I want to create a simple flow that executes a Python script stored in a local directory. The flow will process data fetched from a source directory, then pass it to to a Python script, and generate an output file which will be stored in a target directory.
I tried following something simple that reflects the flow that I want to create, and build upon it, so I used this link to do so: https://community.hortonworks.com/questions/178561/can-anyone-provide-an-example-of-a-python-script-...
. The flow, configuration for each processor, and the script ( except for the print statement, which is print("hello")).
However, the ExecuteStreamCommand keeps returning an error "Permission denied". I tried running NiFi : sudo ./nifi start, but the problem persisted. (https://community.hortonworks.com/questions/35453/how-to-run-one-nifi-processor-as-super-user.html)
I found someone asking about restricted components (ExecuteStreamCommand Processor), and giving them access. I noticed my NiFi UI at the time did not have the key icon on the left, nor create policies option under the menu (located at the top right of the UI). As a result, I realized the NiFi instance I was running was not secure:
1. Restricted components link: https://community.hortonworks.com/questions/114819/required-permission-for-some-nifi-processes.html
2. Policy change link: https://community.hortonworks.com/questions/83999/nifi-access-policies.html
I learned how to run a secure NiFi instance through these three tutorials, using NiFi Registry and NiFi Toolkit 1.8.0:
1. Securing and running NiFi-Registry 0.3.0 (https://www.youtube.com/watch?v=qD03ao3R-a4)
2. Securing, linking and running NiFi 1.8.0 (https://www.youtube.com/watch?v=DSO12fhnZ90)
I only have one user, which is at the admin level. I did not create a test user when I followed the two videos.
I started NiFi 1.8.0 again after the videos, and now I had the policy icons in my UI. Also, in the workflow.png, you can see on the top right of the UI 'CN=sys_admin', which I believe it means that I opened NiFi as the sys_admin user that I've created. I created the same flow as before, and this time, I changed the policies for the admin, and included the admin under 'access restricted components' & 'regardless of restrictions'. Just in case, I added the admin under each sub policy under 'access restricted components'. The problem still persists.
Whats the cause behind the error? And how can I resolve this issue? Any help and suggestions are appreciated.
I was able to solve the issue by running the command 'chmod +x sample.py'. NiFi was able to run the external Python script afterwards.
Thanks for providing the solution, I am facing the same issue with respect to permission,
sorry if this is a silly question, but where did you run this command 'chmod +x sample.py' ?
@Saisreenath According to this article: you should be in the folder containing the script you want to execute to set the execution permissions.
Nifi needs permissions to access the script. It also needs permission to be able to execute the command you gave it. In your error it is saying it cannot run the python command. You may need to adjust that command path. In summary make sure you give correct permissions to everything you put in Properties tab of the NiFi Processor, and use the correct paths for python.
If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.