Thanks mburgess! This was helpful. ... View more

Thanks Shu, that was helpful! ... View more

I was having similar issue, where I had around 7000+ flow files and I needed to merge them into one. Provided that, I didn't know what will be the exact number of flow files, I couldn't specify that number in Minimum Number of Entries attribute of mergeContent processor to make it only 1 flow file after merge. After trying a lot of things, including trying to count the flow files or induce a delay before sending them off to merge, I finally found the solution on the mergeContent documentation below : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache.nifi.processors.standard.MergeContent/index.html#:~:text=The%20%27Defragment%27%20algorithm%20combines%20fragments%20that%20are%20associated%20by%20attributes%20back%20into%20a%20single%20cohesive%20FlowFile.%20The%20%27Bin%2DPacking%20Algorithm%27%20generates%20a%20FlowFile%20populated%20by%20arbitrarily%20chosen%20FlowFiles And there you go ! setting 'Merge Strategy' property to 'Defragment' and 'Attribute Strategy' property to 'Keep Only Common Attributes' fixes the problem for me. In all my flow files, they have common attributes and are now merged into a single flow file. The original question is from 2017, but I hope it helps out somebody else looking for an answer. You can check the original answer here https://stackoverflow.com/questions/56356164/nifi-merging-all-of-necessary-flowfile-in-one-shot-with-mergecontent-processor/79257994#79257994 ... View more

Hi @SS_Jin , Glad to hear that my post helped. Its really hard to suggest something specially when I dont have all the details of what you are trying to do but from what I read, I think Join\Fork Enrichment would work better in these scenarios. The mergerecord way could be problematic when you are reading multiple sources and multiple CSVs where merge behavior can be unpredictable. Also depending on what type of enrichment you are trying to do and how complex its but if you have one to one mapping between record in the DB vs CSV and you are trying to override some data or add new one then you might also consider the LookupRecord processor to simplify your data flow where you dont have to use branching to read and then merge the different sources which might endup saving you some overhead. https://community.cloudera.com/t5/Community-Articles/Data-flow-enrichment-with-NiFi-part-1-LookupRecord-processor/ta-p/246940   ... View more

That's perfect ! ... View more

I set the Translate Field Names to false, it started working, before this it was converting the primary key field to upper case, removing the underscore, so when I run the putSQL, it was throwing error, working fine now, thanks to your answer! ... View more