question Re: How to MergeContent with single header from multiple files.? in Archives of Support Questions (Read Only)

How to MergeContent with single header from multiple files.?

saikrishna_tara — Wed, 10 Aug 2016 02:41:49 GMT

I am using MergeContent processor to Merge 20 small .csv files in to one file. All of them are similar with same header.

The files are getting merged successfully but when i opened the destination file i am seeing 20 headers (one header for each file). is there some setting that i can do where i can have header only once in the Merged file.?

Re: How to MergeContent with single header from multiple files.?

mpayne — Wed, 10 Aug 2016 02:52:03 GMT

MergeContent by itself doesn't allow you to merge parts of the FlowFiles together. However, an option that you can use is to put RouteText before MergeContent. In RouteText, set the Matching Strategy to "Satisfies Expression." Then, add a property named "header" with a value of "${lineNo:equals(1)}". Leave all others as default values. This will route the first line of each FlowFile to the "header" relationship.

You can then auto-terminate the "header" relationship and route "unmatched" to MergeContent. That is, filter out the header line and route everything else to Merge Content.

With MergeContent, you can then set "Header" to the text of your header and set "Delimiter Strategy" to "Text." This will cause MergeContent to add that header line back to the merged FlowFile for you.

Sorry that this is so non-trivial, but I think this approach will at least give you what you're looking for.

Re: How to MergeContent with single header from multiple files.?

saikrishna_tara — Wed, 10 Aug 2016 03:41:23 GMT

Awesome ,thanks. that worked.

Re: How to MergeContent with single header from multiple files.?

saikrishna_tara — Thu, 25 Aug 2016 03:47:16 GMT

@mpayne, is there a way i can check for headers from CVS files and merge all files with similar headers and route other files to failures.??

Re: How to MergeContent with single header from multiple files.?

gauravj1391 — Wed, 01 Mar 2017 19:58:41 GMT

Can't understand. Can you send overall source code for it.

Re: How to MergeContent with single header from multiple files.?

cvsing — Mon, 27 Apr 2020 19:57:30 GMT

The answer @mpayne is correct. Only that setting the header in MergeContent doesn't include a line break at the end of header and the records. So as EL is supported please include line break at the end.