Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to Merge files together by file attribute in Nifi.?

avatar
Super Collaborator

Hi ,I have 1000 files in a folder , file names have date(yyyymmdd) in it. for ex data_20160810.csv...i have 200 files per each day (so 5 days files). I want merge those files by date . so if successful, i my output folder will contain 5 merged files one for each day.

i am trying to do this by "Correlation Attribute Name" property..but still it is not merging in groups.

what am i doing wrong.?

6527-mergebydate.png

1 ACCEPTED SOLUTION

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
11 REPLIES 11

avatar
New Contributor

Using the below configuration

93759-update-attribute.jpg

93760-mergecontent.jpg

avatar
Super Collaborator

@Bryan Bende ,

RouteOnAttribute will know the value of header.? Don't i have to read the value of first line to know the header.? if i have to read first line then should i use SplitText which i read creates a flow file for each line which i want to avoid as i am trying to move thousands of files . below is a simple flow ..i created 5 files , 4 with header foo and one with a different header. i want to merge all the 4 files with foo as header. tried the approach suggested by you above , but its routing everything to unmatched.

6984-filefilter.png

6985-filefilter2.png