Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

How to make sure that all parts of one flow file was merged together?

avatar
Contributor

So I am  using SplitXml processor to split flowfile. After some transformations I am merging those flowfiles back together using MrgeContent. After mergning I have noticed that not all of the flow files merged together. e.g. I have splitted my flowfile  on 100 parts and after merging I've got 2 files with 50 records of my initial flow file each. When i set  Run schedule option to 10 seconds instead of 1 it merged all files together. But what if i don't know how many flowfiles i will get after split and how much time it will take for merge processor to merge it?  I use flowfile name as Correlation attribute name.

 

Thank you  beforehand!  

1 ACCEPTED SOLUTION

avatar
Master Collaborator

When using SplitXML to split the flowfiles, SpilitXML writes the details of the following attributes in each split flow files,  fragment. identifier, fragment.index, fragment.count, segment.original.filename if needed to be merged later then MrgeContent should be configured with Merge Strategy : Defragment "The 'Defragment' algorithm combines fragments that are associated by attributes back into a single cohesive FlowFile" 

Are you doing the same ? with MergeContent?

 

If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Chandan 

View solution in original post

2 REPLIES 2

avatar
Master Collaborator

When using SplitXML to split the flowfiles, SpilitXML writes the details of the following attributes in each split flow files,  fragment. identifier, fragment.index, fragment.count, segment.original.filename if needed to be merged later then MrgeContent should be configured with Merge Strategy : Defragment "The 'Defragment' algorithm combines fragments that are associated by attributes back into a single cohesive FlowFile" 

Are you doing the same ? with MergeContent?

 

If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Chandan 

avatar
Contributor

Thank you kinds sir! My problem was solved!!