https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289881#M214513 <P>Hi Matt,&nbsp;</P><P>I am able to trigger python script now but stuck on how to pass the output file to STDout so that it can come back to flowfile. Here is what I am trying.&nbsp;STDIN seems to be working fine but STDOUT is not working</P><P>&nbsp;</P><P>f = sys.stdin<BR />reader = csv.reader(f)</P><P>***********</P><P>##Python code for creating the output file and passing it to STDOUT</P><P>xmlfile = open("C:\\Output_from_python\\output_file.xml",'w')<BR />xmlFile.write(doc.toprettyxml(indent = '\t'))<BR />xmlFile.close()<BR />sys.stdout = xmlFile</P> Sun, 16 Feb 2020 21:11:15 GMT vikrant_kumar24 2020-02-16T21:11:15Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289679#M214406 <P>I am getting below errors while trying to trigger a python script using ExecuteStreamCommand Processor in NiFi. Attaching the screenshot of the properties of the processor. Please help</P> <P>&nbsp;</P> <P>Below is the error in bulletin board<span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Bulletin Board.PNG" style="width: 999px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/26353i370362B8A50ECFFE/image-size/large?v=v2&amp;px=999" role="button" title="Bulletin Board.PNG" alt="Bulletin Board.PNG" /></span><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="ExecuteStreamCommand.PNG" style="width: 608px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/26351i5CB61816957331AE/image-size/large?v=v2&amp;px=999" role="button" title="ExecuteStreamCommand.PNG" alt="ExecuteStreamCommand.PNG" /></span><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Properties ExecuteStreamCommand.PNG" style="width: 999px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/26352iCD499D8F744BB773/image-size/large?v=v2&amp;px=999" role="button" title="Properties ExecuteStreamCommand.PNG" alt="Properties ExecuteStreamCommand.PNG" /></span></P> <DIV class="bulletin"> <DIV class="bulletin-info"> <DIV class="bulletin-timestamp">23:21:24 CST</DIV> <DIV class="bulletin-severity bulletin-error">ERROR</DIV> <DIV class="bulletin-source bulletin-link">3bc05a73-0170-1000-6f23-80001cc130eb</DIV> <DIV class="clear">&nbsp;</DIV> </DIV> <PRE>ExecuteStreamCommand[id=3bc05a73-0170-1000-6f23-80001cc130eb] Failed to process session due to java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application: org.apache.nifi.processor.exception.ProcessException: java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application</PRE> <DIV class="clear">&nbsp;</DIV> </DIV> <DIV class="bulletin"> <DIV class="bulletin-info"> <DIV class="bulletin-timestamp">23:21:24 CST</DIV> <DIV class="bulletin-severity bulletin-error">ERROR</DIV> <DIV class="bulletin-source bulletin-link">3bc05a73-0170-1000-6f23-80001cc130eb</DIV> <DIV class="clear">&nbsp;</DIV> </DIV> <PRE>ExecuteStreamCommand[id=3bc05a73-0170-1000-6f23-80001cc130eb] Could not create external process to run command: java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application</PRE> <DIV class="clear">&nbsp;</DIV> </DIV> <DIV class="bulletin"> <DIV class="bulletin-info"> <DIV class="bulletin-timestamp">23:21:24 CST</DIV> <DIV class="bulletin-severity bulletin-error">ERROR</DIV> <DIV class="bulletin-source bulletin-link">3bc05a73-0170-1000-6f23-80001cc130eb</DIV> <DIV class="clear">&nbsp;</DIV> </DIV> <PRE>ExecuteStreamCommand[id=3bc05a73-0170-1000-6f23-80001cc130eb] Failed to process session due to java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application: org.apache.nifi.processor.exception.ProcessException: java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application</PRE> <DIV class="clear">&nbsp;</DIV> </DIV> <DIV class="bulletin"> <DIV class="bulletin-info"> <DIV class="bulletin-timestamp">23:21:24 CST</DIV> <DIV class="bulletin-severity bulletin-error">ERROR</DIV> <DIV class="bulletin-source bulletin-link">3bc05a73-0170-1000-6f23-80001cc130eb</DIV> <DIV class="clear">&nbsp;</DIV> </DIV> <PRE>ExecuteStreamCommand[id=3bc05a73-0170-1000-6f23-80001cc130eb] Could not create external process to run command: java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application</PRE> <DIV class="clear">&nbsp;</DIV> </DIV> <DIV class="bulletin"> <DIV class="bulletin-info"> <DIV class="bulletin-timestamp">23:21:24 CST</DIV> <DIV class="bulletin-severity bulletin-error">ERROR</DIV> <DIV class="bulletin-source bulletin-link">3bc05a73-0170-1000-6f23-80001cc130eb</DIV> <DIV class="clear">&nbsp;</DIV> </DIV> <PRE>ExecuteStreamCommand[id=3bc05a73-0170-1000-6f23-80001cc130eb] Failed to process session due to java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application: org.apache.nifi.processor.exception.ProcessException: java.io.IOException: Cannot run program "C:\Users\503067805\Desktop\IDP_Project\POC_Well_Path\Trigger_Python_POC\Python\csv_convertion_final.py": CreateProcess error=193, %1 is not a valid Win32 application</PRE> </DIV> Thu, 13 Feb 2020 07:34:27 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289679#M214406 vikrant_kumar24 2020-02-13T07:34:27Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289724#M214422 <P><a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/63257">@vikrant_kumar24</a>&nbsp;</P><P>&nbsp;</P><P>If you open a terminal window on your Windows server, can you execute your python script?<BR />If you are not successful from command line, the ExecuteStreamCommand processor is not going to be successful as well.</P><P>&nbsp;</P><P>Thanks,</P><P>Matt</P> Thu, 13 Feb 2020 13:38:07 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289724#M214422 MattWho 2020-02-13T13:38:07Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289746#M214438 <P>Hi Mat I can execute the script from spyder (Anaconda) and it is successful. From executestreamcommand processor its not even triggering the script it seems. Its a .py file in the folder. Can this be an issue?</P> Thu, 13 Feb 2020 20:07:30 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289746#M214438 vikrant_kumar24 2020-02-13T20:07:30Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289826#M214488 <P><a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/63257">@vikrant_kumar24</a>&nbsp;</P><P>&nbsp;</P><P>The ExecuteStreamCommand processor is not doing anything fancy here.&nbsp; It is just trying to execute the command you enter.&nbsp; Assuming the command was "myscript.py", The executeStreamCommand would be doing same thing as you would if you opened a terminal window on your NiFi node and clicked on "myscript.py" to run it.&nbsp; Keep in mind that NiFi is performing this action as the user that owns the NiFi java process.&nbsp; The ExecuteStreamCommand processor will pass the FlowFiles content (if there is any) to stdin.</P><P>&nbsp;</P><P>Have you perhaps looked at the ExecuteScript processor instead?</P><P>&nbsp;</P><P>Hope this helps,</P><P>Matt</P> Fri, 14 Feb 2020 17:55:41 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289826#M214488 MattWho 2020-02-14T17:55:41Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289857#M214498 <P>Hi Matt,</P><P>I am able to execute python script from command line. Executescript processor is not an option here because it is an experimental process. I am looking for something more robust which can handle heavy data and also I should be able to put the flow in production. Can you please see the settings in my flow and see if I am doing anything wrong here?</P> Sat, 15 Feb 2020 09:09:27 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289857#M214498 vikrant_kumar24 2020-02-15T09:09:27Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289881#M214513 <P>Hi Matt,&nbsp;</P><P>I am able to trigger python script now but stuck on how to pass the output file to STDout so that it can come back to flowfile. Here is what I am trying.&nbsp;STDIN seems to be working fine but STDOUT is not working</P><P>&nbsp;</P><P>f = sys.stdin<BR />reader = csv.reader(f)</P><P>***********</P><P>##Python code for creating the output file and passing it to STDOUT</P><P>xmlfile = open("C:\\Output_from_python\\output_file.xml",'w')<BR />xmlFile.write(doc.toprettyxml(indent = '\t'))<BR />xmlFile.close()<BR />sys.stdout = xmlFile</P> Sun, 16 Feb 2020 21:11:15 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/289881#M214513 vikrant_kumar24 2020-02-16T21:11:15Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/291293#M215403 <P><a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/63257">@vikrant_kumar24</a>&nbsp;<BR /><BR />You would not configure your python script to write to an XML file on disk</P><P>&nbsp;</P><P><SPAN>NiFi handles the FlowFile creation in the framework. Any data passed by your Python script to&nbsp;</SPAN><EM>STDOUT</EM><SPAN>&nbsp;will be populated into the content of the resulting flowfile passed to the&nbsp;</SPAN><EM>output stream</EM><SPAN>&nbsp;relationship of the&nbsp;</SPAN>ExecuteStreamCommand<SPAN>&nbsp;processor. Your script does not need to have any awareness of what. FlowFile is or how it is created.</SPAN></P><P>&nbsp;</P><P><SPAN>So you simply have your python script send the XML content to STDOUT and NiFi will take care of putting that content in to the FlowFile that will be produced and routed to the "output.stream" relationship of the processor.&nbsp; You can then use the updateAttribute processor the change the filename associated with that content.</SPAN></P><P>&nbsp;</P><P><SPAN>Hope this helps,</SPAN></P><P><SPAN>Matt</SPAN></P> Fri, 06 Mar 2020 18:41:55 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/291293#M215403 MattWho 2020-03-06T18:41:55Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/327613#M230107 <P><a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/63257">@vikrant_kumar24</a>&nbsp; How were you able to run python script in ExecuteStreamCommand? Because I'm also getting the same error.</P> Thu, 14 Oct 2021 05:52:50 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/327613#M230107 krishna123 2021-10-14T05:52:50Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/328142#M230204 <P><a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/63257">@vikrant_kumar24</a>&nbsp;<BR /><BR /></P><P>The ExecuteScript processor has been around for over 6 years as part of Apache NiFi.&nbsp; It has had many improvements and bug fixes over those years just like many other well used components.&nbsp; I'd be reluctant from calling it "experimental" any longer regardless of what the embedded Apache NiFi docs say.<BR /><BR />The only thing to note here is that the ExecuteScript processor does not really execute the "Python" script engine.&nbsp; It is executing "Jython" instead which is a Java implementation of Python.&nbsp; Jython is not 100% compatible with Python, so you must test you script thoroughly.&nbsp;<BR /><BR />Thanks,</P><P>Matt</P> Tue, 19 Oct 2021 18:32:41 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/328142#M230204 MattWho 2021-10-19T18:32:41Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/363550#M239019 <P>I am having the same problem with head as the command, -1 as the argument, and csv in the content of my flowfile. I read someplace that nifi "lost the state"; what does that mean? I have the flowfile, it has content. Why can't the ESC processor send the first line of the file to the attribute I name?&nbsp;</P> Thu, 09 Feb 2023 23:15:27 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/363550#M239019 lt56 2023-02-09T23:15:27Z https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/375345#M242410 <P>I was able to resolve this error, by making the below configuration to ExecuteStreamCommand:</P><P><SPAN>Command Path: &lt;complete path of python.exe&gt;</SPAN></P><P><SPAN>Command Arguments: &lt;complete path of your script&gt;;&lt;other arg&gt;;&lt;other arg&gt;</SPAN></P> Fri, 18 Aug 2023 06:17:40 GMT https://community.cloudera.com/t5/Support-Questions/NiFi-Getting-error-in-ExecuteStreamCommand-while-executing/m-p/375345#M242410 Mamatha 2023-08-18T06:17:40Z