I have a scenario where I'm having trouble on configuring/finding the right processors to update the CSV file.
Scenario: I have a CSV file and it needs to be updated based on some conditions.
Here records will be in millions. Due to this I cannot hold all the records in a processor and write at once.
Example flow: GetFile -> SplitRecord -> SplitJSON -> PutMongo -> (Update CSV File)
The scenario is to create the CSV file. I tried with the 'MergeRecord'. But it is not merging all the records. I can see that, there are 'only 2 options (i.e., 1. Keep all unique attributes & 2. Keep only common attributes). The data may vary for the columns but column names are same for all the records. Here the problem is that 'MergeRecord' is not merging all the records irrespective of columns/data, it is merging few records and few are not. Is there any processor which can merge the JSON records and give the CSV files irrespective of the record contents?