Apache NiFi 1.10 : How to execute a python script as a processor

salvob14 — Mon, 16 Dec 2019 17:47:26 GMT

Disclaimer: crosspost from
https://stackoverflow.com/questions/59359158/apache-nifi-1-10-how-to-execute-a-python-script-as-a-processor

In NiFi <= v1.9 I used the following NiFi setup:

`GetTwitter` generates a JSON for every tweet gathered and attached to it there used to be a `ExecuteStreamCommand` that has the following parameters:

This would allow me to get the flowfile as input, and I could analyse the content and do something (in this specific case I would put the content on Elasticsearch) - You could reach the same result if you do something like this `cat flowfile.json | python <scriptname>.py`

But now in >= v1.10 it seems I cannot do the same. it gives me some error about parameters (it seems it wants to use the flowfile as parameter on calling the command/script instead, as it used to be, pass the flowfile as input.
It gives me the following error:

2019-12-16 14:28:44,953 ERROR [Timer-Driven Process Thread-2] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=ef1d15fb-016e-1000-3050-35fba1733f97] Transferring flow file StandardFlowFileRecord[uuid=c485d8aa-58e9-428c-84b1-b4acbd6275a7,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1576505352573-22, container=default, section=22], offset=663999, length=-1],offset=0,name=242802f9-5ffb-4a2c-90a5-27736459dfd0.json,size=0] to nonzero status. Executable command bash ended in an error: bash: line 1: $'{delete:status:id:1005156317567188992}\r': command not found 2019-12-16 14:28:44,962 ERROR [Timer-Driven Process Thread-2] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=ef1d15fb-016e-1000-3050-35fba1733f97] Transferring flow file StandardFlowFileRecord[uuid=5b617840-ba8e-47b3-9a27-06e8ec0f88bd,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1576505352573-22, container=default, section=22], offset=663999, length=-1],offset=0,name=eb9e236a-1ccb-4243-aa1c-e5de43746f80.json,size=0] to nonzero status. Executable command bash ended in an error: bash: line 1: $'{delete:status:id:1139832600422277120}\r': command not found

It seems it takes the flowfile JSON as parameter instead of input .
Any ideas?

Re: Apache NiFi 1.10 : How to execute a python script as a processor

salvob14 — Tue, 17 Dec 2019 08:16:57 GMT

well, This is not really a problem with NiFi per se:
NiFi was installed with docker so I kept using the docker host as it was the correct machine.

Embarassing.. 🙂

question Apache NiFi 1.10 : How to execute a python script as a processor in Support Questions

Apache NiFi 1.10 : How to execute a python script as a processor

Re: Apache NiFi 1.10 : How to execute a python script as a processor