Support Questions
Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Redirect Error Stream missing in ExecuteStreamCommand

New Contributor


Just started exploring Nifi to automate some of our data analytics flows. I am executing a spark-submit command from shell script using ExecuteStreamCommand processor and need to read the output from the shell script to check if it is success of failure. I think the output is getting redirected to STDERR as a result of which I can see the truncated output in the attribute 'execution.error'. The data is getting truncated as spark output can be huge. In 'ExecuteProcessor' there is a property 'Redirect Error Stream' which is putting all the output in flow file and works for me but I can connect upstream connections to Execute Processor. Planning to send the output to a file from shell script and read the status from there or see if I can automate extracting spark application id from nifi attribute and get logs from yarn, checking if there is any better solution which I may be missing.




New Contributor


I am facing the same issue. Did you get any solution for your problem?



I was also facing a similar kind of issue this code helped me!

Try this link:

It might help you!