Flat file DataFlow connection reference
Learn about the fields used to create a Flat file connection with ThoughtSpot DataFlow.
Here is a list of the fields for a Flat file connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.
Connection properties
- Connection name
-
Name your connection.
Mandatory field.
Example:
FilesConnection
- Connection type
-
Choose the Files connection type.
Mandatory field.
Example:
Files
- File location
-
Specify the base location of the file on the server.
Mandatory field.
Example:
/home/admin/dataflow/data
- Files on remote location
-
Specify If the files on remote server.
Mandatory field.
Default:
No
Other notes:
Advanced configuration
- Protocol
-
Select the required remote server connection.
Mandatory field.
For remote location files only.
Example:
FTP
Valid Values:
FTP, SFTP
Default:
FTP
- Authentication type
-
Specify the authentication type for SFTP Protocol.
Mandatory field.
For SFTP protocol only
Example:
Password
Valid Values:
Key, Password
Default:
Password
- Host
-
Specify the Hostname or the IP address of the remote server.
Mandatory field.
For remote location files only.
Example:
www.example.com
- Port
-
Specify the Port to connect the remote server.
Mandatory field.
For remote location files only.
Example:
1234
- User
-
Specify the user to connect to remote server. This user must have data access privileges.
Mandatory field.
For remote location files only.
Example:
userdi
- Password
-
Specify the password.
Optional field.
For remote location files only, when using password authentication.
Example:
pswrd234%!
- Key file
-
Specify the key file and its fully qualified path.
Mandatory field.
For remote location files only, when using key authentication.
Example:
/app/key/sftp.key
- Passphrase for key file
-
Specify the passphrase for the key file.
Optional field.
For remote location files only, when using key authentication.
Example:
!hello
Sync properties
- File name
-
Specify name of the file.
Mandatory field.
Example:
CUSTOMER.csv
Valid Values:
Any string literal
Default:
The name of the file
Other notes:
To specify wildcard, use '*' character in the file name
- Column delimiter
-
Specify the column delimiter character.
Mandatory field.
Example:
,
Valid Values:
Any printable ASCII character or decimal value for ASCII character
Default:
The delimiter specified in sync
- Enable archive on success
-
Specify if system is to archive file after successful sync.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
- Delete file on success
-
Specify if system to delete file after successful sync.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
- Compression
-
Specify if file uses compression, and compression type.
Optional field.
Example:
gzip
Valid Values:
None, gzip
Default:
None
- Skip header rows
-
Specify the number of header rows to skip when loading data.
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
- Enclosing character
-
Specify if the text columns in the source data needs to be enclosed in quotes.
Optional field.
Example:
Single Valid Values:
Single, Double, Empty
Default:
Empty
- Escape character
-
Specify the escape character if using a text qualifier in the source data.
Optional field.
Example:
\\
Valid Values:
Any ASCII character
Default:
Empty
- Null value
-
Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target.
Optional field.
Example:
+ NULL
+ Valid Values:
+ Any string literal
+ Default:
+ NULL
- Date style
-
Specifies how to interpret the date format.
Optional field.
Example:
YMD
Valid Values:
YMD,MDY,DMY,DMONY,MONDY,Y2MD,MDY2,DMY2,DMONY2, andMONDY2Default:
YMDOther notes:
MDY: 2-digit month, 2-digit day, 4-digit yearDMY: 2-digit month, 2-digit day, 4-digit yearDMONY: 2-digit day, 3-character month name, 4-digit yearMONDY: 3-character month name, 2-digit day, 4-digit yearY2MD: 2-digit year, 2-digit month, 2-digit dayMDY2: 2-digit month, 2-digit day, 2-digit yearDMY2: 2-digit day, 2-digit month, 2-digit yearDMONY2: 2-digit day, 3-character month name, 2-digit yearMONDY2: 3-character month name, 2-digit day, 2-digit year
- Date delimiter
-
Specifies the separator used in the date format.
Optional field.
Example:
-
Valid Values:
Any printable ASCII character
Default:
-
-
- Time style
-
Specifies the format of the time portion in the data.
Optional field.
Example:
24HOUR
Valid Values:
12 Hour, 24 Hour
Default:
24HOUR
- Time delimiter
-
Specifies the character used as separate the time components.
Optional field.
Example:
:
Valid Values:
Any printable ASCII character
Default:
:
- Statistics
-
Specify if the file source count needs to be displayed in Monitor statistics.
Optional field.
Example:
ON
Valid Values:
OFF
Default:
ON
- Skip trailer rows
-
Skip trailer row while processing the data.
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
- Reader encoder
-
Specify the encoding format of the source system.
Optional field.
Example:
UFT-8
Valid Values:
ISO-8859-1, UTF-16, UTF-16BE, WINDOWS-1252
Default:
UFT-8
- TS load options
-
Specifies the parameters passed with the
tsloadcommand, in addition to the commands already included by the application. The format for these parameters is:--<param_1_name> <optional_param_1_value>--<param_2_name> <optional_param_2_value>Optional field.
Example:
--max_ignored_rows 0Valid Values:--user "dbuser"--password "$DIWD"--target_database "ditest"--target_schema "falcon_schema"Default:
--max_ignored_rows 0
- Boolean representation
-
Specifies the representation of data in the boolean field.
Optional field.
Example:
true_false
Valid Values:
true_false, T_F, 1_0, T_NULL
Default:
true_false
Related information
