Table of contents

Manage images

A Watson Studio Local administrator can create a base image that everyone in the cluster can use. Images contain a specific runtime (for example, a Jupyter notebook, RStudio, or Zeppelin notebook), common packages, and shared libraries.

In the Admin Console, click the menu icon ( The menu icon) and click Image Management to augment the runtime images that Watson Studio Local users can get started from.

When the Watson Studio Local administrator deploys them, users can conveniently select them from the Environments page of their project without having to upload any of the packages and libraries themselves.

Want to see image management in action? Watch this short video:
Figure 1. Video that shows how to manage images in Watson Studio Local.

Complete the following steps to customize a runtime image:

  1. Download an existing image
  2. Modify the existing image
  3. Upload the customized image
  4. Approve the customized image
  5. Create a worker environment for the customized image
  6. Delete the customized image
  7. Upload a custom image to Watson Machine Learning and schedule a project release job for it

You can also Manage customized images

Download an existing image

In the Image Management page, go to the Image List and click Download next to the runtime image you want to customize. Wait several minutes for Watson Studio Local to prepare and download the TAR.GZ file.

Image Management screencap

Modify the existing image

Complete the following steps to modify the runtime image to add packages:

  1. Install Docker. See Download Docker Community Edition for details.
  2. Change the working directory to the folder with the downloaded image.
  3. Load the image:
    docker load -i <image_name>_<tag>.tar.gz
    You will receive either the image ID or name:tag identifier.
  4. Create a file named Dockerfile and use the loaded image as the base image. Examples to install the package:
    • Python package in a Jupyter or Zeppelin notebook for pip:
      FROM <identifier>
      RUN /opt/conda/bin/pip install arrow
    • Python package in a Jupyter or Zeppelin notebook for conda (requires admin access):
      FROM <identifier>
      RUN /opt/conda/bin/conda install arrow
    • R package in a Jupyter notebook:
      FROM <identifier>
      RUN Rscript -e "install.packages('data.table',
      repos='http://Rdatatable.github.io/data.table')"
    • For Scala JAR package in a Jupyter or Zeppelin notebook, use the wget, curl, or apt-get command to download the necessary JAR files. After download, move the JAR file to /usr/local/spark/jars. The JAR file will become available to Scala programs.
    • R package in RStudio:
      FROM <identifier>
      RUN Rscript -e "install.packages('data.table',
      repos='http://Rdatatable.github.io/data.table')"
    • Installing R packages is not supported for Zeppelin.
  5. Build the image with a new identifier. Jupyter example (do not forget the last dot):
    docker build -t modified-jupyter:v1.0 .
  6. Save the image and compress it as a tar file. Example:
    docker save modified-jupyter:v1.0 | gzip > modified-jupyter-v1.0.tar.gz

You can now upload the new TAR.GZ file to the Watson Studio Local cluster.

Upload the customized image

Complete the following steps to upload the modified TAR.GZ file back to Watson Studio Local:

  1. In the Image Management page, click Upload runtime image. Image Management screencap
  2. Type in a new image name, tag, and description in the upload page.Upload Image screencap
    • The name should contain only lower-case characters, underscores, or hyphens.
    • The tag should contain only lower-case characters, dots, underscores, or hyphens.
    • The name and tag should be unique and not conflict with any previously uploaded images.
    • The description field should not exceed 256 characters.
  3. Click the Browse button.
  4. Select the TAR.GZ file to be uploaded and click the Upload button. Because runtime image files are large, the upload might take awhile. Note that by default, the server times out in 60 minutes then returns a 502 error.

As a result, all collaborators in Watson Studio Local can select the new image from their project Edit Environment page in the Images pull-down menu. By default, the image will be marked Pending Review.

Runtime environments screencap

Approve the customized image

For informational purposes, a Watson Studio Local administrator can click Approve next to a modified image to mark that it has been successfully tested and approved by other Watson Studio Local users. The Watson Studio Local administrator can also invalidate the image. Only modified images can have their validation states changed, and invalidated images can still be run by Watson Studio Local users.

Image Management

When an image is approved, the following columns change in the image list:

  • Approved: indicates the current state of the image regarding approval (true or false).
  • Validation Change User: indicates the Admin who initiated the latest approval request.
  • Validation Change Date: indicates the timestamp when the latest approval change was made.

Also, Watson Studio Local users can see an image's approval status from their list of runtime environments.

Create a worker environment for the customized image

To edit a worker environment to use the customized image:

  1. Go to the Jobs page.
  2. Click the Workers tab.
  3. Edit the worker environment and select the customized image.
  4. Save the worker.

As a result, whenever Watson Studio Local users run a batch score or evaluate job, they can select this modified worker in the Advanced settings.

Delete the customized image

A Watson Studio Local administrator can delete a non-IBM image (that is not running) by clicking Delete next to it. Watson Studio Local then prompts for confirmation to delete it.

Manage customized images

From the Image Management panel, a Watson Studio Local administrator can see, manage, or validate all images including the ones created by Watson Studio Local users.

The Watson Studio Local administrator can also define a .condarc file (used by all images) to control the source of the packages.

Upload a custom image to Watson Machine Learning and schedule a project release job for it

In a scenario where Watson Machine Learning and Watson Studio Local are hosted on different clusters, and the Watson Studio Local administrator would like to select a custom image that has an updated conda environment for a project release job, the Watson Studio Local administrator can complete the following steps:

  1. In the Admin Console on the Watson Studio Local cluster, download the custom image.
  2. In the Admin Console on the Watson Machine Learning cluster, upload the custom image.
  3. In the Watson Machine Learning client, create a project release.
  4. Edit the worker to select the custom image for it.

    Project release workers page

    Project release Edit worker

  5. Create a job, and configure it to run using the worker. Note that because the worker is shared between all jobs in a project release, editing a worker will change it for all jobs in that release.