Amazon S3 DataFlow connection reference

Learn about the fields used to create an Amazon S3 connection with ThoughtSpot DataFlow.

Here is a list of the fields for an Amazon S3 connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.

Connection properties

Connection name

Name your connection.

Mandatory field.

Example:

AmazonS3Connection

Connection type

Choose the Amazon S3 connection type.

Mandatory field.

Example:

Amazon S3

Amazon S3 URL

Specify the Amazon S3 hostname link.

Mandatory field.

Example:

https://example1-amazonaws.com

Bucket

Specify the bucket.

An Amazon S3 bucket name is globally unique, and the namespace is shared by all AWS accounts.

Mandatory field.

Example:

bucket

Access key

Specify the access key ID generated when creating AWS security credentials.

Mandatory field.

Example:

access key

Secret key

Specify the secrete access key generated when creating AWS security credentials.

Mandatory field.

Example:

ABCDEFGH245HIJK

Sync Properties

File name

Specify the name of the file.

Mandatory field.

Example:

PRODUCT.csv

Valid Values:

Any string literal

Default:

File name for sync.

Other notes:

To specify wildcard pattern, use *.

Enable archive on success

Specify if must archive file after operation succeeds.

Optional field.

Example:

No Valid Values:

Yes, No

Default:

No

Delete file on success

Specify if must delete file after operation succeeds.

Optional field.

Example:

No

Valid Values:
Yes, No

Default:

No

Column delimiter

Specify the column delimiter character.

Mandatory field.

Example:

,

Valid Values:

Any printable ASCII character or decimal value for ASCII character

Default:

1

Skip header rows

Skip the specified number of header rows when loading data.

Optional field.

Example:

5

Valid Values:

Any numeric value

Default:

0

Compression

Specify if the file has compression, and type of compression.

Mandatory field.

Example:

gzip

Valid Values:

None, gzip

Default:

None

Row delimiter

Specify the end of the row character in the extracted data.

Optional field.

Example:

\\n

Valid Values:

Any printable ASCII character

Default:

\\n (new line character)

Enclosing character

Specify if text columns in the source data use enclosing quotation marks, and their type.

Optional field.

Example:

Single

Valid Values:

Single, Double, Empty

Default:

None

Escape character

Specify the escape character if using a text qualifier in the source data.

Optional field.

Example:

\\

Valid Values:

Any ASCII character

Default:

None

Null value

Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target.

Optional field.

Example:

NULL

Valid Values:

Any string literal

Default:

NULL

Date style

Specifies how to interpret the date format.

Optional field.

Example:

YMD

Valid Values:

YMD, MDY, DMY, DMONY, MONDY, Y2MD, MDY2, DMY2, DMONY2, and MONDY2

Default:

YMD

Other notes:

MDY: 2-digit month, 2-digit day, 4-digit year

DMY: 2-digit month, 2-digit day, 4-digit year

DMONY: 2-digit day, 3-character month name, 4-digit year

MONDY: 3-character month name, 2-digit day, 4-digit year

Y2MD: 2-digit year, 2-digit month, 2-digit day

MDY2: 2-digit month, 2-digit day, 2-digit year

DMY2: 2-digit day, 2-digit month, 2-digit year

DMONY2: 2-digit day, 3-character month name, 2-digit year

MONDY2: 3-character month name, 2-digit day, 2-digit year

Date delimiter

Specifies the separator used in the date format.

Optional field.

Example:

  • Valid Values:

    Any printable ASCII character

    Default:

    -

Time style

Specifies the format of the time portion in the data.

Optional field.

Example:

24 hour

Valid Values:

12 Hour, 24 Hour

Default:

24 Hour

Time delimiter

Specifies the character used as separate the time components.

Optional field.

Example:

:

Valid Values:

Any printable ASCII character

Default:

:

Skip trailer rows

Skip the number of trailer rows specified while loading the data.

Optional field.

Example:

5

Valid Values:

Any numeric value

Default:

0

TS load options

Specifies the parameters passed with the tsload command, in addition to the commands already included by the application. The format for these parameters is:

--<param_1_name> <optional_param_1_value>

--<param_2_name> <optional_param_2_value>

Optional field.

Example:

--max_ignored_rows 0

Valid Values:

` --null_value ""`

` --escape_character ""`

` --max_ignored_rows 0`

Default:

--max_ignored_rows 0

Boolean representation

Specifies the representation of data in the boolean field.

Optional field.

Example:

true_false

Valid Values:

true_false, T_F, 1_0, T_NULL

Default:

true_false

Related information

Was this page helpful?
×