https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-possible-for-ExecuteStreamCommand-to-run-a-long/m-p/148087#M48574 <P>There is not such a mechanism in place for ExecuteStreamCommand. ExecuteProcess does have the ability to batch but based on your mention of incoming event rate, seems like the need for handling input which ExecuteProcess does not provide.</P><P>Depending on the nature of your parsing/process, it may be possible to convert this to use InvokeScriptedProcessor which could tie said parser/process to the component lifecycle.</P> Tue, 13 Dec 2016 00:34:56 GMT apiri 2016-12-13T00:34:56Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-possible-for-ExecuteStreamCommand-to-run-a-long/m-p/148085#M48572 <P>I have a very high incoming event rate and a parser written in Python. I'm able to run it with ExecuteStreamCommand, but the latency for starting/stopping a process for every incoming FlowFile is too high and I cant' keep up with my source.</P><P>Is it possible for ExecuteStreamCommand to keep the external process alive and pass new FlowFiles after some signal?</P> Mon, 12 Dec 2016 14:51:47 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-possible-for-ExecuteStreamCommand-to-run-a-long/m-p/148085#M48572 rgelhausen 2016-12-12T14:51:47Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-possible-for-ExecuteStreamCommand-to-run-a-long/m-p/148086#M48573 <P>Hi @Randy Gelhausen,</P><P>Is that parser written in Python using any specific module which is not supported by Jython? If parser is completely compatible with Jython, then "ExecuteScript" processor would work without overhead of spawning a new process for every incoming flowfile.</P> Mon, 12 Dec 2016 23:35:38 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-possible-for-ExecuteStreamCommand-to-run-a-long/m-p/148086#M48573 ninad_patkhedka 2016-12-12T23:35:38Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-possible-for-ExecuteStreamCommand-to-run-a-long/m-p/148087#M48574 <P>There is not such a mechanism in place for ExecuteStreamCommand. ExecuteProcess does have the ability to batch but based on your mention of incoming event rate, seems like the need for handling input which ExecuteProcess does not provide.</P><P>Depending on the nature of your parsing/process, it may be possible to convert this to use InvokeScriptedProcessor which could tie said parser/process to the component lifecycle.</P> Tue, 13 Dec 2016 00:34:56 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-possible-for-ExecuteStreamCommand-to-run-a-long/m-p/148087#M48574 apiri 2016-12-13T00:34:56Z