Properties supported in this sink are shown below ( * indicates required fields )
Property
Description
Name *
Name of the data sink
Description
Description of the data sink
Processing Mode
Select for batch and un-select for streaming. If 'Batch' is selected the value of the switch is set to true. If 'Streaming' is selected the value of the switch is set to false.Default: true
Select Fields / Columns
Comma separated list of fields / columns to select from inputs to the sinkDefault: *
Distinct Values
Select rows with distinct column valuesDefault: false
Table *
Database schema and table nameExample: ${DATABASE}.${TABLENAME}
Output Mode
If mode is batch mode, the values should be either of Append or Overwrite. If streaming mode, the values should be append, complete or update.Default: Append
Part Files Per Partition
Number of part files to write per partition column. WARNING: Setting this value may degrade performance drastically. It may also increase memory and CPU resource usage.Example: 10,20,2000
Create If Doesn't Exist
Create the table if it does not already existDefault: false
Path *
Path where the table data is to be savedExample: s3a://[bucketpath]/tablename.parquet