Nifi 2.0 'Processor' is invalid because initializing runtime environment for the Processor

[#1]

Hello everyone, 

I am facing issue with creating an custom python processor in Nifi 2.0.0. It is linux based server with installed java 22 and python 3.9.18. I also set the nifi.properties command to the python3.9.

Currently I am just trying to get the simplest processor such as 

from nifiapi.flowfiletransform import FlowFileTransform, FlowFileTransformResult

class WriteHelloWorld(FlowFileTransform):
    class Java:
        implements = ['org.apache.nifi.python.processor.FlowFileTransform']
    class ProcessorDetails:
        version = '0.0.1-SNAPSHOT'

    def __init__(self, **kwargs):
        pass

    def transform(self, context, flowfile):
        return FlowFileTransformResult(relationship = "success", contents = "Hello World", attributes = {"greeting": "hello"})

This is picked up by nifi and it seems that successfuly but in the GUI the process still has the warning 'Processor' is invalid because initializing runtime environment for the Processor.

From the logs I saw that there is mentioned /work/python/controller folder which is totally missing for me. When I change the __init__ to pass and manually copy the python interpreter to the /work/python/controller then the warning disappear but I get runtime error which can be seen in the log below

I also verified that under the /work/python/extensions/WriteHelloWorld/0.0.1-SNAPSHOT I see correct files with dependency-download.complete with content True and empty env-creation-complete.txt

In bin also I see all venv activate files created.  

I also can manually enter the created venv using the activate.csh file. So the main problem seems to be with the communication between nifi and python. Nifi runs as admin. 

Any idea how to solve it or find the rootcause of this problem? I already did all the steps from Apache NiFi custom processor runtime environment e... - Cloudera Community - 390917

[#2]

Hi @PeterC ,

I dont have experience creating python ext. processor under linux system but I would imagine it will work the same as under windows. I noticed however in python command you are using python9.3 , can you switch this to just python ? You should not be needing to manually creating the folder structure under ./work/python and all dependencies should be downloaded for your.

[#3]

Hi @SAMSAL I have tried with python, python3 and python3.9. Same problem for all. Also could you confirm whether the /work/python/controller should be created and if yes what exactly should be there? 

[#4]

What gets created is ".\work\python\extensions\[CustomProcessor]\2.0.0-SNAPSHOT".
it should have  all downloaded dependencies as specified in your processor code,  then venv files and folder, in case of windows it creates Script folder where all the venv files:

 I think in Unix it store those files under Bin folder but it should not matters.

where you seeing any  error messages or warning initially before manually creating the folder? Also one thing you can try is use later python version than 3.9 , I know in my case 3.11 worked well for me.