Table of contents

Transfer files between DSX Local and HDP cluster

You can transfer files between the HDP cluster and the DSX Local cluster using Python utility functions or from a Python environment within your notebook.

Requirement: The HDP cluster must be set up to work with DSX Local, and DSX Local must be configured to work with the HDP cluster.

Python example

Restriction: The WebHDFS URL must always end in a forward slash, for example, https://9.87.654.321:50070/gateway/dsx/webhdfs/v1/.

To download files from the HDP cluster to DSX Local:

dsx_core_utils.download_hdfs_file("https://9.87.654.321:50070/gateway/dsx/webhdfs/v1/", "/user/user1/sample_07.csv", "/user-home/1001/DSX_Projects/sample_07.txt")

To upload files from DSX Local to the HDP cluster:

dsx_core_utils.upload_hdfs_file("https://9.87.654.321:50070/gateway/dsx/webhdfs/v1/", "/user-home/1001/DSX_Projects/sample_07.txt", "/user/user1/sample_07.csv" )