Support Questions

Find answers, ask questions, and share your expertise

NIFI:Merging Flowfiles by filename in MergeContent processor

avatar
Contributor

I have flowfiles named as (1,3,4,5 and etc) i use this ${filename} attribute for invoking online service, then i got big response and split it line by line but at the end i need to merge my flowfiles based on their name i think mergecontent doesn't work prperly i use Correlation atribute name -filename and i have also increased minimum and maximum number of entries but nothing helped me here is my workflow:

also there is several subject i am interested in:

  1. I think that main reason my mergecontent doesn't work properly is that my file names don't go one after another , can this bean real reason?
  2. Can you reccomend me any better solution for such task?

39435-flow.png

4 REPLIES 4

avatar
Master Guru

@sally sally, if you are having filename attribute then make use RouteonAttribute processor to get same filename files into one MergeContent processor. by this method all the same filename flowfiles comes to one mergecontent processor will help to resolve issues with merging the flowfiles.

RouteonAttribute Properties:-

To get only filename 1 from the flow files

${filename:equals('1')}

Example:-
RouteonAttribute:-

40438-routeonattribute-config.png

Make use of Property 1, 2 and connect 1, 2 relationships to two seperate merge contents..

40439-flow-merge.png

avatar
Contributor

Ok ,but as i guess i will havee to add one routeOnAttriute for each fileaname what can i do if i have more than 100 filenames?

avatar
Contributor

should i change correlation attribute name with something else?( i mean with different attribute)

avatar
Master Guru

@sally sally, i think if you need to merge files based on filename then you have to use those many merge content processors(e.g if 100 filenames you need to have 100 merge contents).
Can you please share more details about your merging strategy on what basis you are merging the flow files is it based on size(or) some thing else? and can also share us the configs that you are using now to merge content based on filename?