Google Cloud Storage DataFlow connection reference
Learn about the fields used to create a Google Cloud Storage connection with ThoughtSpot DataFlow.
Here is a list of the fields for a Google Cloud Storage connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.
Connection properties
- Connection name
-
Name your connection.
Mandatory field.
Example:
GoogleCloudStorageConnection
- Connection type
-
Choose the Google Cloud storage connection type.
Mandatory field.
Example:
Google Cloud storage
- Project ID
-
The identification number given to particular project, always unique.
Mandatory field.
Example:
myproject-1234
- Authentication type
-
It can be either Service Account or Access Tokens.
Mandatory field.
Example:
Service Account
Valid Values:
Service Account, Access Token
Default:
Service Account
- Service account key or access token
-
Provide the Service Account key when authentication type is selected as Service account and token when access token is selected as authentication type.
Mandatory field.
Example:
ABCDEFGH245HIJK
- Bucket
-
To specify GCS bucket name.
Mandatory field.
Example:
bucket
- File location
-
Specify directory of source on agent location.
Mandatory field.
- Use proxy
-
If required, to use a proxy, select the checkbox Use Proxy and provide the details.
Optional field.
Other notes:
These are under Advanced configuration
- Host
-
Specify the hostname or the IP address of the GCS system.
Mandatory field.
Example:
www.example.com
- Port
-
Specify the port associated to the GCS system.
Mandatory field.
Example:
1234
- Protocol
-
It can be either http or https.
Mandatory field.
Example:
http
Valid Values:
http, https
Default:
http
Sync properties
- File name
-
Specify name of the file.
Mandatory field.
Example:
PRODUCT.csv
Valid Values:
Any string literal
Default:
The name of the file used to create sync Other notes:
To specify wildcard pattern use '*' character in the file name
- Enable archive on success
-
Specify if file needs to be archived once it is succeeded.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
- Delete file on success
-
Specify if file needs to be deleted after execution is successful.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
- Column delimiter
-
Specify the column delimiter character.
Mandatory field.
Example:
,
Valid Values:
Any printable ASCII character or decimal value for ASCII character
Default:
The delimiter specified in sync
- Skip header rows
-
Skip the number of header rows specified while loading the data.
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
- Compression
-
Specify this if the file is compressed and what kind of compressed file it is.
Mandatory field.
Example:
gzip
Valid Values:
None, gzip
Default:
None
- Row delimiter
-
Specifies the character to be used to indicate the end of the row in the extracted data.
Optional field.
Example:
\\n
Valid Values:
Any printable ASCII character
Default:
\\n (new line character)
- Enclosing character
-
Specify if the text columns in the source data is enclosed in quotes.
Optional field.
Example:
Single
Valid Values:
Single, Double, Empty
Default:
Empty
- Escape character
-
Specify the escape character if using a text qualifier in the source data.
Optional field.
Example:
\\
Valid Values:
Any ASCII character
Default:
Empty
- Null value
-
Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target.
Optional field.
Example:
NULL
Valid Values:
Any string literal
Default:
NULL
- Date style
-
Specifies how to interpret the date format.
Optional field.
Example:
YMD
Valid Values:
YMD
,MDY
,DMY
,DMONY
,MONDY
,Y2MD
,MDY2
,DMY2
,DMONY2
, andMONDY2
Default:
YMD
Other notes:
MDY
: 2-digit month, 2-digit day, 4-digit yearDMY
: 2-digit month, 2-digit day, 4-digit yearDMONY
: 2-digit day, 3-character month name, 4-digit yearMONDY
: 3-character month name, 2-digit day, 4-digit yearY2MD
: 2-digit year, 2-digit month, 2-digit dayMDY2
: 2-digit month, 2-digit day, 2-digit yearDMY2
: 2-digit day, 2-digit month, 2-digit yearDMONY2
: 2-digit day, 3-character month name, 2-digit yearMONDY2
: 3-character month name, 2-digit day, 2-digit year
- Date delimiter
-
Specifies the separator used in the date format.
Optional field.
Example:
-
Valid Values:
Any printable ASCII character
Default:
-
- Time style
-
Specifies the format of the time portion in the data.
Optional field.
Example:
24 hour
Valid Values:
12 Hour, 24 Hour
Default:
24 Hour
- Time delimiter
-
Specifies the character used as separate the time components.
Optional field.
Example:
:
Valid Values:
Any printable ASCII character
Default:
:
- Decimal delimiter
-
Specify the decimal delimiter for float/double/numeric data types.
Optional field.
Example:
.
Valid Values:
(.), (,), EMPTY
Default:
EMPTY
- Skip trailer rows
-
Skip the number of trailer rows specified while loading the data.
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
- TS load options
-
Specify additional parameters passed with the
tsload
command. The format for these parameters is:--<param_1_name> <optional_param_1_value>
Optional field.
Example:
--max_ignored_rows 0
Valid Values:
--null_value ""
--escape_character ""
--max_ignored_rows 0
Default:
--max_ignored_rows 0
- Boolean representation
-
Specifies the representation of data in the boolean field.
Optional field.
Example:
true_false
Valid Values:
true_false, T_F, 1_0, T_NULL
Default:
true_false
Related information