Properties supported in this sink are shown below ( * indicates required fields )
Property
Description
Name *
Name of the data sink
Description
Description of the data sink
Processing Mode
Select for batch and un-select for streaming. If 'Batch' is selected the value of the switch is set to true. If 'Streaming' is selected the value of the switch is set to false.Default: true
Connection *
Pre-defined Redshift connection
Table *
Table to write to
Output Mode
If mode is batch mode, the values should be either of Append, Overwrite, ErrorIfExists or Ignore. If streaming mode, the values should be append, complete or update.Default: ErrorIfExists
Partition By
Comma separated column names to partition byExample: year, month, day
Forward S3 Credentials
If true then this library will automatically discover the credentials that Spark is using to connect to S3 and will forward those credentials to Redshift over JDBC. These credentials are sent as part of the JDBC query, so therefore it is strongly recommended to enable SSL encryption of the JDBC connection when using this option.Default: false
Select Fields / Columns
Comma separated list of fields / columns to select from inputs to the sinkExample: id, name, city, state, zipDefault: *
Distribution Style
The Redshift Distribution Style to be used when creating a tableDefault: EVEN
Distribution Key
The name of a column in the table to use as the distribution key when creating a tableExample: JOB
Sort Key Spec
Sort Keys supported by RedshiftExample: REGION
Preactions
A semicolon-separated list of SQL commands that are executed before data is transferred between Spark and Snowflake.Example: Select * from EMPLOYEES where ACTIVE = true
Postactions
A semicolon-separated list of SQL commands that are executed after data is transferred between Spark and Snowflake.Example: Select * from EMPLOYEES where ACTIVE = false
Extra Copy Options
A list extra options to append to the Redshift COPY command when loading data, e.g. TRUNCATECOLUMNS or MAXERROR n (see the Redshift docs for other options).Example: TRUNCATECOLUMNS