Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ExecuteStreamCommand getting hung with a shell script which includes spark-sql queries. Once hung I am not able to clear the queue unless I stop & Start nifi.Other processor which can be used??

Highlighted

ExecuteStreamCommand getting hung with a shell script which includes spark-sql queries. Once hung I am not able to clear the queue unless I stop & Start nifi.Other processor which can be used??

New Contributor
 
1 REPLY 1

Re: ExecuteStreamCommand getting hung with a shell script which includes spark-sql queries. Once hung I am not able to clear the queue unless I stop & Start nifi.Other processor which can be used??

Master Guru
@archana k

-

The inability to kill an active thread on a NiFi processors has been a known limitation to NiFi. There has been work done as part of https://issues.apache.org/jira/browse/NIFI-1295 that will allow user to kill these threads through the NiFi UI so you will no longer need to restart NiFi to do so.

-

As far as your script is concerned. The questions is whether your script ever returned correctly to the processor. Getting multiple thread dumps (up to 5 dumps at an interval of 5 minutes) while the processor is in this "hung" state would give you some threads states to compare to see what the processor is doing during that time frame. Sometimes the issue is as easy as to much standard out being spewed out while the script (updating table output for example) is executing. Suppressing this excessive output generally helps with scripts of this nature.

-

To obtain a thread dump from NiFi, simpley execute the following command:

../<path to NiFi>/bin/nifi.sh dump <name of dump file>

-

Thank you,

Matt

-

When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum.

Don't have an account?
Coming from Hortonworks? Activate your account here