Table of contents

Manage packages as a Watson Studio Local user

Watson Studio Local users can add additional packages to a base image (for example, custom, external, or third-party libraries and packages), which can then be reused by other users.

Watson Studio Local includes many preinstalled libraries. Before you install a library, check the list of preinstalled libraries. Run the appropriate command from a notebook cell:

  • Python: !pip list --isolated
  • R: installed.packages()

If the library that you want is not listed, or you want to use a Scala library in a notebook, use the steps in the following sections to install it. The format for library packages depends on the programming language.

To install a Python library

  1. Use the Python pip package installer command to install Python libraries to your notebook. Examples:
    • !pip install prettyplotlib installs the prettyplotlib library into a pod's conda, for instance, /opt/conda/lib/python2.7/site-packages and then removes the package once the pod is terminated.
    • !pip install --user prettyplotlib installs the prettyplotlib package in your user home, for instance /user-home/<uid>/.local/lib/python2.7/site-packages (limited to only one user and requires an environment restart to take effect), and the package exists even after the pod is terminated.
  2. Use the Python import command to import the library components. For example, run the following command in a code cell:
    import prettyplotlib as ppl
  3. Restart the kernel.
To use custom functions defined in Python scripts

You can import a python script to the Scripts section of your project, and then import the functions like any other Python module. The following example imports all functions in a file named dsxl_support_functions.py:

sys.path.insert(0, '../scripts/')
from dsxl_support_functions import *

where ../scripts is the location where you imported your Python scripts. The imported functions can be called like any regular functions defined in modules.

To save libraries and packages in a custom Jupyter image for anyone to use

A Watson Studio Local user can install a library or package in the root conda environment, and save the running environment as an image so that it can be shared with other users.

Requirement: The Watson Studio Local cluster must have access to either an external or internal package repository. If the cluster does not have access, the Watson Studio Local administrator must create the custom image.

Complete the following steps:

  1. Open a notebook or terminal in the running Jupyter environment. You can open a terminal from the Launch Terminal icon ( Launch Terminal).
  2. Install the library or package using conda, for example, conda install -y arrow.
  3. Go to Environments and click the Save icon (Save) for the running environment to save it into a new custom image on the cluster.
  4. Name and tag the new custom image. Click the Save button.

To view or delete this new custom image, click the menu icon ( The menu icon) in the Watson Studio Local client and click My Images.

Any Watson Studio Local user can select this new image by clicking Edit settings next to the corresponding environment. From the admin console, the Watson Studio Local administrator can see, manage, or validate the images created by Watson Studio Local users from the Manage images panel.

To load an R package

  1. Use the R install.packages() function to install new R packages. For example, run the following command in a code cell to install the ggplot2 package for plotting functions:
    install.packages("ggplot2")
    The imported package can be used by all R notebooks running in the Spark service.
  2. Use the R library() function to load the installed package. For example, run the following command in a code cell:
    library("ggplot2")
    You can now call plotting functions from the ggplot2 package in your notebook.

To use a Scala library

Libraries for Scala notebooks are typically packaged as Java™ archive (JAR) files.

To cache a library temporarily
The libraries for a Scala notebook are not installed to the Spark service. Instead they are cached when they are downloaded and are only available for the time that the notebook runs.
  • To use a single library without dependencies, from a public web server:
    1. Locate the publicly available URL to the library that you want to install. If you create a custom library, you can post it to any publicly available repository, such as GitHub.
    2. Download the library you want to use in your notebook by running the following command in a code cell:
      %AddJar URL_to_jar_file
  • To use a library with dependencies, from a public Maven repository:
    1. Add and import a library with all its dependencies by running the following command. You need the groupId, artifactId, and version of the dependency. For example:
      %AddDeps org.apache.spark
      spark-streaming-kafka_2.10 1.1.0 --transitive

Learn more