My SplitFile processor connected (follows by) ValidateRecords processor. ValidateRecords use CSVReader with following configuration: Input_schema is avro: {
"type": "record",
"name": "geo_data",
"fields": [
{
"name": "id",
"type": [
"int",
"null"
]
},
{
"name": "description",
"type": [
"string",
"null"
]
}
]
} And the source of this pipeline in ListS3 and FetchS3 processors.
... View more
