question Re: how to add output stream relationship result of Execute Stream Command processor into original flow file in Support Questions

how to add output stream relationship result of Execute Stream Command processor into original flow file

Techie123 — Tue, 14 Jun 2022 12:23:39 GMT

I need a help, I am new to NiFi. I am using python script. I wanted to add output stream relationship result of ExecuteStreamCommand processor into original flow file. In original file I am getting body and the script is generating users information. Please help me

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

MattWho — Tue, 14 Jun 2022 12:57:55 GMT

@Techie123

The ExecuteStreamCommand processor is working as designed:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.16.2/org.apache.nifi.processors.standard.ExecuteStreamCommand/index.html

Executes an external command on the contents of a flow file, and creates a new flow file with the results of the command.

You could route both the "original" and "output stream " relationships via the same outbound connection to a mergeContent processor which can merge the content from both source FlowFiles into a single FlowFile.
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.16.2/org.apache.nifi.processors.standard.MergeContent/index.html

If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

SAMSAL — Tue, 14 Jun 2022 13:01:58 GMT

Hi,

Not sure I fully understand your question. Can you please provide screenshot of the flow you have and explain the expected output of each processor? From what I can understand you want to keep the original file but somehow you want to attach the output of the command stream to it. There is property on the ExecuteCommandStream Processor called "Output Destination Attribute" where you can store the output stream as an attribute on the original flow file. if you want to modify the content of the flow file after that based on the new attribute then you can use ExecuteScript Processor.

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

Techie123 — Tue, 14 Jun 2022 13:29:58 GMT

Yes. But the thing is script is generating token decryption result and original flow file contains body. But the merge content processor is merging flow files randomly

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

Techie123 — Tue, 14 Jun 2022 13:32:46 GMT

Hi @SAMSAL , it should not return as an attribute but the data, because the output stream contains decrypted data and the original flow file also has the some important data

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

Techie123 — Tue, 14 Jun 2022 13:35:36 GMT

@MattWho , can we append output stream result to original flow file without using mergeContent processor because the we have multiple incoming flow files and i just wanted to append two at a time, original flow file with the output stream result

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

SAMSAL — Tue, 14 Jun 2022 14:00:42 GMT

Is there a unique attribute on the original flowfile (like filename) that gets set on the output stream ? if so, you can set the mergeContent processor property "Correlation Attribute Name" to the attribute containing the unique value that is shared between original and output, this way you guarantee that only related original and output are getting merged.

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

Techie123 — Tue, 14 Jun 2022 14:21:44 GMT

@SAMSAL , thanks for quick response. But I am new to nifi, can you please explain it a bit. you want me to add the filename explicitly? and then merged these flow files?

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

SAMSAL — Tue, 14 Jun 2022 14:42:32 GMT

how is the original flowfile is generated. usually you should have filename attribute set by default on the original flowfile. this should be unique for the flowfile and this filename could be passed to the output stream, can you check that?

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

Techie123 — Tue, 14 Jun 2022 15:37:50 GMT

Hi @SAMSAL , I am doing the same but still not getting the desired output.

Can you refer snapshot, Am i missing something?

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

SAMSAL — Tue, 14 Jun 2022 17:06:14 GMT

first you need to check if there is an attribute called filename on the original flowfile and its the same passed the output stream flowfile. If there is, then you should specify it as "filename" in the merge content property and NOT as "${filename}".

Re: how to add output stream relationship result of Execute Stream Command processor into original flow file

Techie123 — Wed, 15 Jun 2022 03:35:15 GMT

Hi @SAMSAL, thanks a lot for you help. I'm getting data as expected.