Salesforce

Salesforce database data source

Property
Description
Name *
Name of the data sink
Description
Description of the data sink
Processing Mode
Select for batch and un-select for streaming. If 'Batch' is selected the value of the switch is set to true. If 'Streaming' is selected the value of the switch is set to false.Default: true
Connection *
Pre-defined Salesforce connection
Infer Schema
Infers the input schema automatically from data. Requires one extra pass over the data.Default: false
Schema
Source schema to assist during the design of the pipeline
Table or Query
Salesforce table that should be read OR a query that will be used to read data from the Salesforce source.
Filter Expression
WHERE clause allows for filtering records, with support for SOQL Date functions. Additionally, DataMorph Date functions are available to specify time windows using predefined substitutions.\nFor instance, if you want to retrieve data modified in the last hour, you can use the following syntax:
lastModifiedDate > ${DM_CURRENT_TIME_MINUS_HOURS_6}
Where DM_CURRENT_TIME_MINUS_HOURS_N represents the past N hours, with N ranging from 1 to 23.
More functions will be released in future updates.To filter records where the LastModifiedDate is today and the hour of the day is greater than 5:
LastModifiedDate = TODAY and
HOUR_IN_DAY(LastModifiedDate) > 5.
To filter records between specific CreatedDate ranges:
CreatedDate > 2024-11-25T21:32:19.000Z and
CreatedDate < 2024-11-26T21:32:19.000Z
Normalize Column Names
Normalizes column names by replacing special characters ,;{}()&/\\n\\t= and space with the given string
Query Timeout in Seconds
The number of seconds the driver will wait for a Statement object to execute to the given number of seconds. Zero means there is no limit.Default: 0
Number of Rows Fetched per Round Trip
The fetch size, which determines how many rows to fetch per round trip.Default: 0
Cache
MEMORY_ONLY: Persist data in memory only in deserialized formatMEMORY_AND_DISK: Persist data in memory and if enough memory is not available evicted blocks will be stored on diskMEMORY_ONLY_SER: Same as MEMORY_ONLY but difference being it persists in serialized format. This is generally more space-efficient than deserialized format, but more CPU-intensive to read.MEMORY_AND_DISK_SER: Same as MEMORY_AND_DISK storage level difference being it persists in serialized formatDISK_ONLY: Persist the data partitions only on diskMEMORY_ONLY_2, MEMORY_AND_DISK_2: Same as the levels above, but replicate each partition on two cluster nodesOFF_HEAP: Similar to MEMORY_ONLY_SER, but store the data in off-heap memory. This requires off-heap memory to be enabledDefault: NONE