Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Keep updating the existing CSV file

Keep updating the existing CSV file

New Contributor

I have a scenario where I'm having trouble on configuring/finding the right processors to update the CSV file.
Scenario: I have a CSV file and it needs to be updated based on some conditions.
Here records will be in millions. Due to this I cannot hold all the records in a processor and write at once.
Example flow: GetFile -> SplitRecord -> SplitJSON -> PutMongo -> (Update CSV File)

2 REPLIES 2

Re: Keep updating the existing CSV file

Super Guru

@Raju Chigicherla

If you want to split the csv file based on some conditions then use QueryRecord (or) RouteText processors then by using either of Update Record (or) LookUp Record processors to update the contents of flowfile.

Re: Keep updating the existing CSV file

New Contributor

@Shu
The scenario is to create the CSV file. I tried with the 'MergeRecord'. But it is not merging all the records. I can see that, there are 'only 2 options (i.e., 1. Keep all unique attributes & 2. Keep only common attributes). The data may vary for the columns but column names are same for all the records. Here the problem is that 'MergeRecord' is not merging all the records irrespective of columns/data, it is merging few records and few are not. Is there any processor which can merge the JSON records and give the CSV files irrespective of the record contents?