Is it possible to combine two flows with different content using MergeRecord?

[#1]

Hi,

I have the following problem - I have a flow in which I have a json file, then via updateAttribute I give this flow some id and then I divide this flow into two success paths. The json file stays on one of them, on the other I perform some operations, I clear the contest and I have nothing there except the attributes I need. Now I would like to combine these paths using MergeRecord and use the id I gave at the beginning as the correlation attribute. Is it possible to connect json file that I have in one path with the empty content that I have in the other? Because I only care about the attributes from the second path and I noticed that in the case of MergeRecord, it is merged only in the case of the same content when selecting Bin-Packing Algorithm as the merge strategy. Is there any way around this?

[#2]

Hi @MWM ,

The following worked for me:

The GenerateFlwoFile has json content and an attribute flowfile_id with value 123

In the ReplaceText I replace everything with empty string:

In the UpdateAttribute Im adding new Attribute new_attr with value of 555

The MergeConent is configured as follows:

I'm using flowfile_id as Correlation Attribute Name. Also notice how I set the "Minimum Number of Entries" to 2 so that original flowfile will wait until the second  ready.

The result is the original flowfile content with the new added attribute.

Another option to MergeRecord is you can use PutDistributedMapCache and FetchDistributedMapCache to store the original content into Cache, then do whatever is needed to get the new attributes  and finally fetch the original content again , this will give you the original flowfile including the new attributes. The only caveat with this approach that you have to create two controller services:  DistributedMapCacheClientService & DistributedMapCacheServer.  Another issue with this DistributedMapCacheClientService  is that you have to provide a server hostname which could be the same as your nifi node, however this produces a single point of failure specially when you have cluster. For more info:

https://stackoverflow.com/questions/44590296/how-does-one-setup-a-distributed-map-cache-for-nifi

If that helps please accept solution.

Thanks