Table of contents

Set up runtime environments

To avoid slow compute performance within your project runtime environments, you can reserve CPU and memory resources in advance from the Environments page in your project. Note that reserving CPU and memory reduces the availability of CPU and memory for other runtime environments.

Caution: Turning Reserve resources off can affect system performance if available CPU and RAM on the servers become overcommitted.

Runtime environments

A runtime environment represents an allocation of compute resource (one or more docker containers) on the DSX Local cluster. You can define multiple environments for specific images such as RStudio and notebooks. To change your CPU and memory allocations, click Edit settings next to the environment. Tip: if you create a dataframe that loads a large data file in your notebook and receive an error that the kernel died, then you can edit the environment to increase memory.

Runtime environment

If your notebook crashes or the spark context becomes unavailable (sc undefined in the notebook), click Stop next to the environment and return to the notebook to get the spark context working again.

Tip: If you allocate the maximum CPU for a runtime environment, the environment might stay in Pending state indefinitely. As a workaround, reduce the CPU allocation.

To view all runtime environments in the DSX system, click Manage across projects or go to the Environments page from the menu icon (The menu icon).