- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to Merge files together by file attribute in Nifi.?
- Labels:
-
Apache NiFi
Created on ‎08-10-2016 07:57 PM - edited ‎08-18-2019 03:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi ,I have 1000 files in a folder , file names have date(yyyymmdd) in it. for ex data_20160810.csv...i have 200 files per each day (so 5 days files). I want merge those files by date . so if successful, i my output folder will contain 5 merged files one for each day.
i am trying to do this by "Correlation Attribute Name" property..but still it is not merging in groups.
what am i doing wrong.?
Created ‎08-10-2016 09:32 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The attribute to correlate on needs to be present in the flowfile for the Merge processor to use it. If you are using FetchFile to get the file, you can add an attribute into that processor using the filename or the substring of the file name. Then it will be present in the flowfile for subsequent processors to use.
Created on ‎11-30-2018 07:56 AM - edited ‎08-18-2019 03:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Using the below configuration
Created on ‎08-25-2016 08:07 PM - edited ‎08-18-2019 03:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
RouteOnAttribute will know the value of header.? Don't i have to read the value of first line to know the header.? if i have to read first line then should i use SplitText which i read creates a flow file for each line which i want to avoid as i am trying to move thousands of files . below is a simple flow ..i created 5 files , 4 with header foo and one with a different header. i want to merge all the 4 files with foo as header. tried the approach suggested by you above , but its routing everything to unmatched.

- « Previous
-
- 1
- 2
- Next »