Support Questions
Find answers, ask questions, and share your expertise
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

Keep updating the existing CSV file

New Contributor

I have a scenario where I'm having trouble on configuring/finding the right processors to update the CSV file.
Scenario: I have a CSV file and it needs to be updated based on some conditions.
Here records will be in millions. Due to this I cannot hold all the records in a processor and write at once.
Example flow: GetFile -> SplitRecord -> SplitJSON -> PutMongo -> (Update CSV File)


Super Guru

@Raju Chigicherla

If you want to split the csv file based on some conditions then use QueryRecord (or) RouteText processors then by using either of Update Record (or) LookUp Record processors to update the contents of flowfile.

New Contributor

The scenario is to create the CSV file. I tried with the 'MergeRecord'. But it is not merging all the records. I can see that, there are 'only 2 options (i.e., 1. Keep all unique attributes & 2. Keep only common attributes). The data may vary for the columns but column names are same for all the records. Here the problem is that 'MergeRecord' is not merging all the records irrespective of columns/data, it is merging few records and few are not. Is there any processor which can merge the JSON records and give the CSV files irrespective of the record contents?