Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NIFI add new column dynamically to CSV

Highlighted

NIFI add new column dynamically to CSV

New Contributor

Hi,

My requirement is to add new columns to existing CSV with header. CSV files in folder having difffernt schema but additional columns are static based on the filename(Getting part of file names seperated by underscore)


I was looking at below link but avroschemaregistry is defined with userdefined schema. In my case , output schema is decided based on input schema that changes frequently based on filenames.

https://community.hortonworks.com/questions/212355/extract-values-from-csv-and-place-it-in-a-new-col...


Any solution on how to do this.

Input Data1:

Userid|address|state


Input Data2:

Userid|cobra|eventid|insured|lastvisit


For both of these input data, I have to add additonal columns like filename, dept


I have different inputs(around 100 files having different formats). I cannot keep writing the schemas in avroschema registry as this schema gets changed. May I know how to achieve this.


Thanks,

Khaja