Merge attributes from different flowfiles

[#1]

Hi Team,

I am stuck at one situation.could you please help/suggest with the below scenario?

i want to merge attributes of two NiFi flowfiles into one flowfile. say for example, i have a flowfile with 5 attributes coming from processor A and another flowfile with 10 attributes coming from processor B.Now, i want to have a single flowfile with 15 attributes(from A and B processor). Is there any way, we can do this in NiFi or any way to append these attributes into one another.

Thanks in advance,

Wasim

[#2]

wasimakram054

You can use ReplaceText processor to combine All the 15 attributes together and the result will be a single flowfile .

Example in ReplaceText processor as below 

$attr1,$attr2,...$attr15 in replace text .

You would need to link two flowfile queues to ReplaceText procssor and add the above example condition .

[#3]

@adhishankarit 

This only works if intent is to replace or append the FlowFile Attributes to the content of the FlowFile.  
But you will still not result in a single FlowFiles will all 15 attributes.  ReplaceText does not merge anything and only has access to FlowFile Attributes on the FlowFile that is being executed upon.  

You would still need to merge the content of those 2 FlowFiles to have a single FlowFiles with all 15 attributes which would exist in the content of the new FlowFile.

[#4]

@wasimakram 

First we need to answer some questions here:
1.What about the content of these two FlowFiles?  Do both have same content?  How do you want to handle that when you merge these FlowFIles?
2.  Are all 15 of these attributes unique?  Meaning that the same attribute name does not exist on both FlowFiles.

Scenario 1: Let's assume both FlowFiles have same content.
1. Then you could user modifyBytes processor to remove all content from only one of those FlowFIles.
2. Then use Merge Content processor to merge those two FlowFiles and set property Keep all unique attributes.  This will result in one FlowFile with content from only FlowFile that had content and all unique attributes from both source FlowFiles set on output new FlowFile.

Scenario 2:  Each FlowFile has unique content.
1. You could still use MergeContent just as in Scenario one and the resulting FlowFile will have all teh unique attributes and the merged content from both source FlowFiles. 

Another option:
You could use maybe the putDistributedMapCache processor to write the desired attributes to a cache server.  Then use the FetchDistributedMapCache to consume the needed attributes and place them on the other FlowFile.  This will not perform as well and you need to consider under volume that you only process one set of FlowFiles at a time.  Can be done but not as performant and adds complexity to dataflow design.

Hope this helps,
Matt