Using Databricks Delta Lake utilities

You can use the Databricks Delta Lake client to customize connections in DataFlow.

Requirements to install Databricks Delta Lake

  • python 3- 3.6 and above

  • python 2- 2.7.9 and above

Follow these steps to install the Databricks Delta Lake client:

  1. Run pip install databricks-cli using the appropriate version of pip for your Python installation.

    pip install databricks-cli --user

  2. In case if pip is not present then please use pip3 for installation.

    pip3 install databricks-cli --user

  3. Create Symlink of dbfs command

    sudo ln -snf /home/admin/.local/bin/dbfs /usr/bin/dbfs

  4. Before running the CLI commands, you must set up authentication. To authenticate to the CLI we use a personal access token. We have REST API to get access token.

    Curl -H 'Authorization: Bearer AUTH-TOKEN -X POST -d '{ "lifetime_seconds": 100, "comment": "Remotely genarated token" }' https://DBFS-HOST/api/2.0/token/create

You must authenticate, to get access to Databricks REST APIs. At least one UI generated token required to generate other tokens. In this command, replace AUTH-TOKEN with UI Generated token and DBFS-HOST with databricks host.

Restart the DataFlow service.

#tscli --adv service restart dataflow
Was this page helpful?
×