Table of contents

The Jupyter and Spark notebook environment

Notebooks for Jupyter run on Jupyter kernels and, if the notebook uses Spark APIs, those kernels run in Spark engines.

You can learn to use Spark in Watson Studio Local by opening any of several sample notebooks, such as:

  • Learn the basics about notebooks and Apache Spark
  • Use Spark for Python to load data and run SQL queries
  • Use Spark for R to load data and run SQL queries
  • Use Spark for Scala to load data and run SQL queries

Jupyter kernels

When you open a notebook in edit mode, exactly one interactive session connects to a Jupyter kernel for the notebook language and Spark version that you select. This kernel executes code that you send and returns the computational results. To change the kernel to a new notebook language or Spark mode (such as local or cluster), click Kernel > Change kernel.

Example of Change kernel

If necessary, you can restart or reconnect to the kernel. When you restart a kernel, the kernel is stopped and then started with the same session, but all execution results are lost. When you reconnect to a kernel after losing a connection, the notebook is connected to the same kernel session, and all previous execution results are available.

The kernel remains active even if you leave the notebook or close the web browser window. When you reopen the same notebook, the notebook is connected to the same kernel.