Table of contents

Host scripts as web services in DSX Local

Starting in Version 1.2, you can host scripts as a web service in DSX Local.

When you create or import a script in DSX Local, you can test and run functions present in the script, which are exposed are RESTful endpoints. By invoking a curl command, you can reach the corresponding Kubernetes artifact on which the script is hosted and submit a POST request to run a function by passing all relevant arguments that the function expects.

  • Support for Python 2.7, Python 3.5, and R scripts.
  • An Interactive Development Environment (IDE) for scripts. Clicking the name of the script brings up the IDE. The following actions can be found in the action bar of the IDE:
    • Save the script after it is edited.
    • Run the script as a job.
    • Expose the script as a web service.
    • Edit run configuration for job runs.
    • See job run history.
    • Alter IDE's font size and tab spacing. Turn on/off auto-completion. Turn on/off auto-save.
    • Information panel
  • An action labeled Test script as API in the pull-down for each script in the scripts grid.
  • Python scripts when requested to be test as an API are hosted in the project's Jupyter-2 or Jupyter-3 containers. The container depends on what you select and the type of the script.
  • R scripts when requested to be test as an API are hosted in the project's Rstudio container.

When you select Test script as API, a modal loads to start the respective containers.

After the container is up and running, the modal displays two curl commands that you can run from any console. The first command, helps you identify your access token, which expires every 13 hours.

curl -k -X GET "<your_cluster_ip>/v1/preauth/validateAuth" -u $username:$password

Replace $username and $password with your login credentials. Look for the accessToken attribute, and copy its value (without the double quotation marks). You use the attribute in the second curl command.

You can then invoke the second curl command, which queries the web service and helps run the function that you specify in the command. You also provide the access token that you received earlier to this command so that you are authenticated before the web service can begin running the function.

curl -k -X POST "<your_cluster_ip>/dsx-py2-script/ibmdsxuser-1001/1524173487793/$function" -H 'Content-Type: application/json' -H "Authorization: Bearer $accessToken" -d '{ "relativeScriptPath": "scripts/test.py", "args": $args }'

Replace $function with the name of the function you want to run. Replace $accessToken with the value you copied. Replace $args with arguments that the function expects. relativeScriptPath is pre-populated. No changes that are required to that.

Important:

  • If your script contains a function that is named init, it is seen as a special function by the web service. In DSX Local, the HTTP service that runs the user-requested function also runs the init method at every request. The init method helps in the models use-case for pre-loading the model into memory before querying the generated script for accuracy and scoring results. Users are advised to have a function with the init signature only if the subsequent functions that the users want to call requires things to be set up in the environment for them to run successfully. Users are not required to have an init method if their custom scripts do not fit the previous use-case. This function if present must take no arguments.
  • Since the arguments to the function are sent with a curl command, one expects it to be a string or a valid JSON. The $args are sent as is to the function. So if you send it as an object (python dictionary), the code in the function is expected to use accurate references to keys and values in the object.
  • The access token expires after 13 hours and a new one is generated. The first curl command helps retrieve the latest and valid token.

Example user script (written in Python):

from __future__ import print_function
import sys

def eprint(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

def add_handler(num1, num2, num3):
    print('STDOUT in add_handler')
    dict = {}
    dict['sum'] = num1 + num2 + num3
    sys.stderr.write('Another STDERR in add_handler')
    return dict

def add(obj):
    print('A sample STDOUT message')
    a = obj['a']
    b = obj['b']
    c = obj['c']
    eprint('Sample STDERR message')

    return add_handler(a, b, c)

def init():
    # Do nothing for now
    pass

Example of the curl command to run the add function:

curl -k -X POST "https://<cluster_ip>/dsx-py2-script/ibmdsxuser-999/1524164032049/add" -H 'Content-Type: application/json' -H "Authorization: Bearer $accessToken" -d '{ "relativeScriptPath": "scripts/test.py", "args": {"a": 1, "b": 2, "c": 3} }'

Note the $args object sent in the curl command and how it gets de-constructed within the add function.

Output from running this command: {"result": {"sum": 6}, "stderr": ["Sample STDERR message", "Another STDERR in add_handler"], "stdout": ["A sample STDOUT message", "STDOUT in add_handler"]}

Understanding the output: The output object (JSON) consists of three attributes:

  • result indicates the value that is returned by the function.
  • stderr is an array of stderr statements that are thrown during the process of sourcing the user script and running the function.
  • stdout is an array of stdout statements that gets run within the function.

You can run just one function at a time. However, this function can call multiple other functions if need be in a nested paradigm.