Flat file connection reference

Learn about the fields used to create a Flat file connection with ThoughtSpot DataFlow.

Here is a list of the fields for a Flat file connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.

Connection properties

Connection name

Name your connection. Mandatory field.

Example:

FilesConnection

Connection type

Choose the Files connection type. Mandatory field.

Example:

Files

File location

Specify the base location of the file on the server. Mandatory field.

Example:

/home/admin/dataflow/data

Files on remote location

Specify if the files on remote server. Mandatory field.

Default:

No

Other notes:

These are under Advanced configuration.

Protocol

Select the required remote server connection. Mandatory field.
For remote location files only.

Example:

FTP

Valid Values:

FTP, SFTP

Default:

FTP

Authentication type

Specify the authentication type for SFTP Protocol. Mandatory field.
For SFTP protocol only.

Example:

Password

Valid Values:

Key, Password

Default:

Password

Host

Specify the Hostname or the IP address of the remote server. Mandatory field.
For remote location files only.

Example:

www.example.com

Port

Specify the Port to connect the remote server. Mandatory field.
For remote location files only.

Example:

1234

User

Specify the user to connect to remote server. This user must have data access privileges. Mandatory field.
For remote location files only.

Example:

userdi

Password

Specify the password for the User.

Optional field.

Example:

pswrd234%!

Note:

For remote location files only, when using password authentication.

Key file

Specify the key file and its fully qualified path. Mandatory field.
For remote location files only, when using key authentication.

Example:

/app/key/sftp.key

Passphrase for key file

Specify the passphrase for the key file. Optional field.
For remote location files only, when using key authentication.

Example:

!hello

Sync properties

File name

Specify name of the file. Mandatory field.

Example:

CUSTOMER.csv

Valid Values:

Any string literal

Default:

The name of the file

Other notes:

To specify wildcard, use '*' character in the file name

Column delimiter

Specify the column delimiter character. Mandatory field.

Example:

,

Valid Values:

Any printable ASCII character or decimal value for ASCII character

Default:

The delimiter specified in sync

Enable archive on success

Specify if system is to archive file after successful sync. Optional field.

Example:

No

Valid Values:

Yes, No

Default:

No

Delete file on success

Specify if system to delete file after successful sync. Optional field.

Example:

No

Valid Values:

Yes, No

Default:

No

Compression

Specify if file uses compression, and compression type. Optional field.

Example:

gzip

Valid Values:

None, gzip

Default:

None

Skip header rows

Specify the number of header rows to skip when loading data. Optional field.

Example:

5

Valid Values:

Any numeric value

Default:

0

Enclosing character

Specify if the text columns in the source data needs to be enclosed in quotes. Optional field.

Example:

Single

Valid Values:

Single, Double, Empty

Default:

Empty

Escape character

Specify the escape character if using a text qualifier in the source data. Optional field.

Example:

\\

Valid Values:

Any ASCII character

Default:

Empty

Null value

Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target. Optional field.

Example:

NULL

Valid Values:

Any string literal

Default:

NULL

Date style

Specifies how to interpret the date format. Optional field.

Example:

YMD

Valid Values:

YMD, MDY, DMY, DMONY, MONDY, Y2MD, MDY2, DMY2, DMONY2, and MONDY2

Default:

YMD

Other notes:

MDY: 2-digit month, 2-digit day, 4-digit year
DMY: 2-digit month, 2-digit day, 4-digit year
DMONY: 2-digit day, 3-character month name, 4-digit year
MONDY: 3-character month name, 2-digit day, 4-digit year
Y2MD: 2-digit year, 2-digit month, 2-digit day
MDY2: 2-digit month, 2-digit day, 2-digit year
DMY2: 2-digit day, 2-digit month, 2-digit year
DMONY2: 2-digit day, 3-character month name, 2-digit year
MONDY2: 3-character month name, 2-digit day, 2-digit year

Date delimiter

Specifies the separator used in the date format. Optional field.

Example:

-

Valid Values:

Any printable ASCII character

Default:

-

Time style

Specifies the format of the time portion in the data. Optional field.

Example:

24HOUR

Valid Values:

12 Hour, 24 Hour

Default:

24HOUR

Time delimiter

Specifies the character used as separate the time components. Optional field.

Example:

:

Valid Values:

Any printable ASCII character

Default:

:

Reader encoder

Specify the encoding format of the source system. Optional field.

Example:

UFT-8

Valid Values:

ISO-8859-1, UTF-16, UTF-16BE, WINDOWS-1252

Default:

UFT-8

TS load options

Specifies the parameters passed with the tsload command, in addition to the commands already included by the application. The format for these parameters is:

--<param_1_name> <optional_param_1_value>
--<param_2_name> <optional_param_2_value>

Optional field.

Example:
--max_ignored_rows 0
Valid Values:
--null_value ""
--escape_character ""
--max_ignored_rows 0
Default:
--max_ignored_rows 0
Boolean representation

Specifies the representation of data in the boolean field. Optional field.

Example:

true_false

Valid Values:

true_false, T_F, 1_0, T_NULL

Default:

true_false

Parallel thread limit

Specifies the number of parallel threads to unload data. Optional field.

Example:

4

Valid Values:

Any numeric value

Default:

1

Max ignored rows

Abort the transaction after encountering 'n' ignored rows. Optional field.

Example:

0

Valid Values:

Any numeric value

Default:

0