Flatten/Explode

Flatten/Explode processor

Properties

Properties supported in this processor are shown below ( * indicates required fields )
Property
Description
Name *
Name of the processor
Description
Description of the processor
Select Fields / Columns
Comma separated fields / columns to select from inputs to the processorDefault: *
Explode Outer
Creates a new row for each element in the given array or map column. If the array/map is null or empty then null is producedDefault: true
Recursive
Select to use EXPLODE columns recursively until all the array, struct and map types are processedDefault: false
Separator
Separator character for generating nested column nameDefault: _
Cache
MEMORY_ONLY: Persist data in memory only in deserialized formatMEMORY_AND_DISK: Persist data in memory and if enough memory is not available evicted blocks will be stored on diskMEMORY_ONLY_SER: Same as MEMORY_ONLY but difference being it persists in serialized format. This is generally more space-efficient than deserialized format, but more CPU-intensive to read.MEMORY_AND_DISK_SER: Same as MEMORY_AND_DISK storage level difference being it persists in serialized formatDISK_ONLY: Persist the data partitions only on diskMEMORY_ONLY_2, MEMORY_AND_DISK_2: Same as the levels above, but replicate each partition on two cluster nodesOFF_HEAP: Similar to MEMORY_ONLY_SER, but store the data in off-heap memory. This requires off-heap memory to be enabledDefault: NONE