Hi, i'm very new to NiFi and i was hoping for some advice. I'm taking in some JSON data which i'm assuming needs to be split before i can successfully convert it to CSV and then merge together again. json.png shows the JSON data before it has been converted, csv.png shows the data after its been converted and merged back together and desiredCSV.png shows what i want the csv file to look like.
I'm currently using SplitJson -> EvaluateJsonPath -> ReplaceText -> MergeContent. Iv'e tried using the ConvertRecord processor but i can't seem to configure it properly.
This is the flow for one record, i then need to do the same thing with over a hundred other records and merge them all together into one big CSV file!?
Any help would be greatly appreciated.
You need to change the
Delimiter Strategy
Text
set Demarcator property value to New line(shift + enter)
Then you can get the merged file contents in new lines.
Output:-
The merged output flowfile will looks like below with the above properties
2018,1,12,3,0,0
2018,1,12,2,0,0
Your flow looks fine to me. The only thing that you need to take care of it merging ALL the fragments while using MergeContent. So use Defragment merge strategy. This will pick ALL the flow files created from a single document by SplitJson and will make sure that all of them are merged before moving further. Follows a sample MergeContent config.
Let know if that works for you or if you need any other help.
PS - If you need any help with your existing flow, refer to this link
Hi Rahul, thanks for your prompt reply. Unfortunately the resulting CSV file is still being merged in the same manner. see following snippet.
as you can see i don't have the heading and all csv files are being appended onto the one line.
You need to change the
Delimiter Strategy
Text
set Demarcator property value to New line(shift + enter)
Then you can get the merged file contents in new lines.
Output:-
The merged output flowfile will looks like below with the above properties
2018,1,12,3,0,0
2018,1,12,2,0,0
