Databricks Delta Lake connection reference

Learn about the fields used to create Databricks Delta Lake connection with ThoughtSpot DataFlow.

Here is a list of the fields for a Databricks Delta Lake connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.

Connection properties

Connection name

Name your connection. Mandatory field.

Example:

AzureDatabricksConnection

Connection type

Choose the Azure Databricks connection type. Mandatory field.

Example:

Azure Databricks

Server Hostname

Specify the hostname of the Databricks server. Mandatory field.

Example:

www.example.com

Port

Specify the port associated with the system. Mandatory field.

Example:

1234

Default:

443

HTTP path

Specify the HTTP Path. Mandatory field.

Example:

abcservice

Protocol

Specify the remote server connection. Mandatory field.

Example:

https

Valid Values:

https, http

Default:

https

Cluster id

Specify the canonical identifier for the cluster. Mandatory field.

Example:

1234

Access token

Specify the access token to authenticate Databricks API. Mandatory field.

Example:

ABCDEFGH245HIJK

DBFS stage location

Specify the mount storage object location. Mandatory field.

Default:

/dataflow/stage

Sync properties

Data extraction mode

Specify the extraction type.

Example:

JDBC

Valid Values:

JDBC, Spark API

Default:

JDBC

Column delimiter

Specify the column delimiter character. Mandatory field.

Example:

1

Valid Values:

Any printable ASCII character or decimal value for ASCII character

Default:

1

Enclosing character

Specify if the text columns in the source data needs to be enclosed in quotes. Optional field.

Example:

DOUBLE

Valid Values:

SINGLE, DOUBLE

Default:

DOUBLE

Other notes:

This is required if the text data has newline character or delimiter character.

Escape character

Specify this if the text qualifier is mentioned. This should be the character which escapes the text qualifier character in the source data. Optional field.

Example:

\"

Valid Values:

Any ASCII character

Default:

\"

Max ignored rows

Abort the transaction after encountering 'n' ignored rows. Optional field.

Example:

0

Valid Values:

Any numeric value

Default:

0

tsload options

Specifies the parameters passed with the tsload command, in addition to the commands already included by the application. The format for these parameters is:

<param_1_name> = <param_1_value>
Example:
date_time_format = %Y-%m-%d
date_format = %Y-%m-%d;time_format = %H:%M:%S
Valid Values:
null_value = NULL
max_ignored_rows = 0
Default:
max_ignored_rows = 0

Related information


Was this page helpful?