Apache NiFi custom processor runtime environment error

[#1]

I am using NiFi 2.0 that supports python scripts directly from a working directory, after I add my custom processor to the canvas, an error "Processor invalid because initializing runtime environment for the processor" is indicated in each custom processor. I have already installed Java JDK 21, python 3.11 and all the requirements specified in the official documentation. How can I resolve this?

I have already installed Java JDK 21, python 3.11 and all the requirements specified in the official documentation.I changed the python extension parameters in the nifi.properties file.

This is what appears in the app-log:
2024-07-24 10:31:06,608 ERROR [Initialize CreateFlowFile] org.apache.nifi.NiFi An Unknown Error Occurred in Thread VirtualThread[#138,Initialize CreateFlowFile]/runnable@ForkJoinPool-1-worker-2
java.lang.RuntimeException: Failed to launch Process for Python Processor [CreateFlowFile] Version [0.0.1-SNAPSHOT]
at org.apache.nifi.py4j.StandardPythonBridge.getProcessForNextComponent(StandardPythonBridge.java:267)
at org.apache.nifi.py4j.StandardPythonBridge.createProcessorBridge(StandardPythonBridge.java:125)
at org.apache.nifi.py4j.StandardPythonBridge.lambda$createProcessor$7(StandardPythonBridge.java:142)
at org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
at java.base/java.lang.VirtualThread.run(VirtualThread.java:309)
Caused by: java.io.IOException: Failed to create Python Environment ./work/python/extensions/CreateFlowFile/0.0.1-SNAPSHOT: process existed with code 1
at org.apache.nifi.py4j.PythonProcess.setupEnvironment(PythonProcess.java:359)
at org.apache.nifi.py4j.PythonProcess.start(PythonProcess.java:129)
at org.apache.nifi.py4j.StandardPythonBridge.getProcessForNextComponent(StandardPythonBridge.java:248)
... 4 common frames omitted

[#2]

@cyrine Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @joseomjr @MattWho  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

Hey,

I have been running into similar issues in recent weeks building custom processors, I have yet to find a satisfactory solution to this issue outside of removing the processors from the python/extensions folder, restarting, and then adding them back one by one.

Sometimes this will clear the issue, sometimes it won't.

Is this a known issue with Nifi 2.0, and is there any workaround or fix that the team is aware of?

Thanks.

[#4]

Hi,

To help troubleshoot :

1- what version of nifi 2.0 are you using exactly and what system?

2- Can you post the error as appears in the log?

3- Do you see any dependency packages under the path /work/python/extension/{processor name}/2.0.0-SNAPSHOT. If the answer is no, can you provide screenshot of what us under that folder.

4- Do you see any file called "env-creation-complete.txt" under the same folder above?

5- Are you getting any error when starting nifi after enabling the python extension by uncommenting the following line :

nifi.python.command=...

Thanks

[#5]

i had the same issue
i made sudo apt install python3-venv
restarted apache nifi
and it worked