Support Questions

Find answers, ask questions, and share your expertise

Redirect Error Stream missing in ExecuteStreamCommand

New Contributor


Just started exploring Nifi to automate some of our data analytics flows. I am executing a spark-submit command from shell script using ExecuteStreamCommand processor and need to read the output from the shell script to check if it is success of failure. I think the output is getting redirected to STDERR as a result of which I can see the truncated output in the attribute 'execution.error'. The data is getting truncated as spark output can be huge. In 'ExecuteProcessor' there is a property 'Redirect Error Stream' which is putting all the output in flow file and works for me but I can connect upstream connections to Execute Processor. Planning to send the output to a file from shell script and read the status from there or see if I can automate extracting spark application id from nifi attribute and get logs from yarn, checking if there is any better solution which I may be missing.




New Contributor


I am facing the same issue. Did you get any solution for your problem?



I was also facing a similar kind of issue this code helped me!

Try this link:

It might help you!