How to use FlowFile attributes from one flow in the another flow

AA24 — Tue, 19 Oct 2021 15:12:39 GMT

I have a requirement of loading Data from MySQL to Salesforce (using Salesforce BLUK API) using Nifi. For that I am first calling a Salesforce SOAP API to get session_id(accessToken), then I am passing this session_id to another REST API to create a BULK Load job on Salesforce and I am extracting the job_id and adding both the session_id and job_id into the attribute of the flow file.

Till now everything was fine, but now I need to extract the data from MySQL Table using QueryDatabaseTable processor which doesn't accepts any upstream link, but I need to use the session_id and job_id to pass it along with the MySQL Data to Salesforce BULK API REST call using InvokeHTTP, so now I have two disconnected flows, one has the job_id and session_id to be passed to Salesforce REST API and the other flow has the Data from MySQL DB, but I am unable to use both of these in the InvokeHTTP. Its a very tricky situation.

Is there any way, that I can store the session_id and job_id from the 1st flow inside the flowfile attributes into some variable and use them while calling the REST API when invoking the REST API along with the Data extracted from MySQL ?

Thanks

AA24

Re: How to use FlowFile attributes from one flow in the another flow

MattWho — Tue, 19 Oct 2021 21:04:57 GMT

@AA24

The easiest way to accomplish this is to use the PutDistributedMapCache processor in one flow to write the attributes values you want to share to a cache server and on your other flow use the FetchDistributedMapCache processor to retrieve those cached attributes and add them to your other FlowFiles that need them.

Another option is to use the MergeContent processor. On flow one where it looks like you are extracting your session_id and job_id you would use the ModfiyBytes processor to zero out the content leaving you with a FlowFile that only has attributes and then use MergeContent to combine this FlowFile with the FlowFile in your second flow. In the MergeContent processor you would configure "Attribute Strategy" to use "Keep All Unique Attributes".

If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt

question How to use FlowFile attributes from one flow in the another flow in Support Questions

How to use FlowFile attributes from one flow in the another flow

Re: How to use FlowFile attributes from one flow in the another flow