Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to MergeContent with single header from multiple files.?

avatar
Super Collaborator

I am using MergeContent processor to Merge 20 small .csv files in to one file. All of them are similar with same header.

The files are getting merged successfully but when i opened the destination file i am seeing 20 headers (one header for each file). is there some setting that i can do where i can have header only once in the Merged file.?

1 ACCEPTED SOLUTION

avatar
Expert Contributor

MergeContent by itself doesn't allow you to merge parts of the FlowFiles together. However, an option that you can use is to put RouteText before MergeContent. In RouteText, set the Matching Strategy to "Satisfies Expression." Then, add a property named "header" with a value of "${lineNo:equals(1)}". Leave all others as default values. This will route the first line of each FlowFile to the "header" relationship.

You can then auto-terminate the "header" relationship and route "unmatched" to MergeContent. That is, filter out the header line and route everything else to Merge Content.

With MergeContent, you can then set "Header" to the text of your header and set "Delimiter Strategy" to "Text." This will cause MergeContent to add that header line back to the merged FlowFile for you.

Sorry that this is so non-trivial, but I think this approach will at least give you what you're looking for.

View solution in original post

5 REPLIES 5

avatar
Expert Contributor

MergeContent by itself doesn't allow you to merge parts of the FlowFiles together. However, an option that you can use is to put RouteText before MergeContent. In RouteText, set the Matching Strategy to "Satisfies Expression." Then, add a property named "header" with a value of "${lineNo:equals(1)}". Leave all others as default values. This will route the first line of each FlowFile to the "header" relationship.

You can then auto-terminate the "header" relationship and route "unmatched" to MergeContent. That is, filter out the header line and route everything else to Merge Content.

With MergeContent, you can then set "Header" to the text of your header and set "Delimiter Strategy" to "Text." This will cause MergeContent to add that header line back to the merged FlowFile for you.

Sorry that this is so non-trivial, but I think this approach will at least give you what you're looking for.

avatar
Contributor

Can't understand. Can you send overall source code for it.

avatar
Explorer

The answer @mpayne is correct. Only that setting the header in MergeContent doesn't include a line break at the end of header and the records. So as EL is supported please include line break at the end.

avatar
Super Collaborator

Awesome ,thanks. that worked.

avatar
Super Collaborator

@mpayne, is there a way i can check for headers from CVS files and merge all files with similar headers and route other files to failures.??