Support Questions
Find answers, ask questions, and share your expertise

NiFi: running python web-scraper through ExecuteCommandStream / ExecuteProcess processors

New Contributor

I fully typed out this question here: https://stackoverflow.com/questions/54450058/nifi-running-python-web-scraper-through-executecommands...

But, the overall gist is that I have a python web-scraping script in my docker container, and I'm trying to have the processor scrape what I need, and send it on down my pipeline.

Problem is, I can't get it to scrape without throwing some "command not found" errors, and I have no idea how to get the system to recognize my python script. Python3 is downloaded in the container. The SO link above fully explains my issue.

I've taken a look at this: https://community.hortonworks.com/questions/178561/can-anyone-provide-an-example-of-a-python-script-..., a good starting place, but not truly germane to the issue.

2 REPLIES 2

Re: NiFi: running python web-scraper through ExecuteCommandStream / ExecuteProcess processors

New Contributor

@Robert Papel

This can be solved with a POST InvokeHTTP processor.

PM for more info...

Re: NiFi: running python web-scraper through ExecuteCommandStream / ExecuteProcess processors

The above question and the entire response thread below was originally posted in the Community Help track. On Sat Jul 6 20:39 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions previously posted to another site.

Bill Brooks, Community Moderator
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.