Python Extension Processors In M3 release still go into invalid state for not finding the bin folder

SAMSAL — Wed, 12 Jun 2024 21:18:37 GMT

Initially I thought this was fixed in M3 and felt quite the relief for not having to manually rename the scripts folder to bin and restarting nifi twice for that: once to create the initial extension folders under the work folder, and second after renaming the scripts to bin. It turns out that it works for the very first time however upon subsequent restarts the python extension processors start going into invalid state. When I check the log I found that its still looking for the bin folder:

2024-06-12 15:58:22,105 ERROR [Initialize ExcelTableToJson] org.apache.nifi.NiFi An Unknown Error Occurred in Thread VirtualThread[#118,Initialize ExcelTableToJson]/runnable@ForkJoinPool-1-worker-2: java.lang.RuntimeException: Failed to launch Process for Python Processor [ExcelTableToJson] Version [2.0.0-SNAPSHOT]
java.lang.RuntimeException: Failed to launch Process for Python Processor [ExcelTableToJson] Version [2.0.0-SNAPSHOT]
	at org.apache.nifi.py4j.StandardPythonBridge.getProcessForNextComponent(StandardPythonBridge.java:262)
	at org.apache.nifi.py4j.StandardPythonBridge.createProcessorBridge(StandardPythonBridge.java:123)
	at org.apache.nifi.py4j.StandardPythonBridge.lambda$createProcessor$7(StandardPythonBridge.java:140)
	at org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
	at java.base/java.lang.VirtualThread.run(VirtualThread.java:309)
Caused by: java.io.IOException: Cannot run program "F:\nifi-2.0.0-M3-test\.\work\python\extensions\ExcelTableToJson\2.0.0-SNAPSHOT\bin\python": CreateProcess error=2, The system cannot find the file specified
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1170)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1089)
	at org.apache.nifi.py4j.PythonProcess.launchPythonProcess(PythonProcess.java:283)
	at org.apache.nifi.py4j.PythonProcess.start(PythonProcess.java:129)
	at org.apache.nifi.py4j.StandardPythonBridge.getProcessForNextComponent(StandardPythonBridge.java:243)
	... 4 common frames omitted
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
	at java.base/java.lang.ProcessImpl.create(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:500)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:159)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1126)
	... 8 common frames omitted
2024-06-12 15:58:22,109 ERROR [Initialize ExcelTableToJson] org.apache.nifi.NiFi An Unknown Error Occurred in Thread VirtualThread[#116,Initialize ExcelTableToJson]/runnable@ForkJoinPool-1-worker-7: java.lang.RuntimeException: Failed to launch Process for Python Processor [ExcelTableToJson] Version [2.0.0-SNAPSHOT]
java.lang.RuntimeException: Failed to launch Process for Python Processor [ExcelTableToJson] Version [2.0.0-SNAPSHOT]
	at org.apache.nifi.py4j.StandardPythonBridge.getProcessForNextComponent(StandardPythonBridge.java:262)
	at org.apache.nifi.py4j.StandardPythonBridge.createProcessorBridge(StandardPythonBridge.java:123)
	at org.apache.nifi.py4j.StandardPythonBridge.lambda$createProcessor$7(StandardPythonBridge.java:140)
	at org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
	at java.base/java.lang.VirtualThread.run(VirtualThread.java:309)
Caused by: java.io.IOException: Cannot run program "F:\nifi-2.0.0-M3-test\.\work\python\extensions\ExcelTableToJson\2.0.0-SNAPSHOT\bin\python": CreateProcess error=2, The system cannot find the file specified
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1170)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1089)
	at org.apache.nifi.py4j.PythonProcess.launchPythonProcess(PythonProcess.java:283)
	at org.apache.nifi.py4j.PythonProcess.start(PythonProcess.java:129)
	at org.apache.nifi.py4j.StandardPythonBridge.getProcessForNextComponent(StandardPythonBridge.java:243)
	... 4 common frames omitted
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
	at java.base/java.lang.ProcessImpl.create(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:500)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:159)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1126)
	... 8 common frames omitted

When rename the scripts folder to bin and restart Nifi again its start working again!

Also Im not sure why there is no separate python.log file anymore like in M1 & M2, instead everything related to python extension is getting logged into the main nifi-app.log which makes it harder to find.

I have already posted a comment in github under pull request 12514 which seem to be related to this issue.

@pvillard

***UPDATE***:

I think I know why this is happening and under what circumstance. I notice that whenever this happens actually the bin folder is getting created upon downloading the dependencies. For example if you look at the log above I'm creating python processor that reads an excel table and converts into array of json records. This processor uses the following dependencies:
['pandas','numpy','openpyxl']
When they get downloaded a bin folder is getting created with the following file insde:
f2py.exe

I think this confuses Nifi and once its find the bin folder it thinks thats where venv files are therefore the processor goes into invalid state when it cant find them there.

That is why I think its better not to hard code the folder path to look for one or the other and depending on what if finds first (which in this case the bin folder) it will start looking there. I think this is better if its left to be configurable depending on which evn Nifi is deployed on.

Thanks

Re: Python Extension Processors In M3 release still go into invalid state for not finding the bin folder

MattWho — Fri, 14 Jun 2024 14:36:46 GMT

@SAMSAL

Looking at output provided, you appear to running your Apache NiFi on Windows. It appears this issue was raised 2 days ago against M3 in Apache Jira here:

https://issues.apache.org/jira/browse/NIFI-13394

It is currently unresolved. You can certainly create an Apache jira account and add additional comments to this jira with your detailed findings.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

question Re: Python Extension Processors In M3 release still go into invalid state for not finding the bin folder in Support Questions

Python Extension Processors In M3 release still go into invalid state for not finding the bin folder

Re: Python Extension Processors In M3 release still go into invalid state for not finding the bin folder