Support Questions

Find answers, ask questions, and share your expertise

Use NiFi to execute Python Script

avatar
New Contributor

My goal want to use Nifi to schedule to run python script like daily, monthly.
My project are gonna be forecasting Which my python script are gonnaload the ML model then query data from data base and use it as input in my ML model then it will make a prediction then save that data into database.

How can I use NiFi to do that? or Should I create API for the Model then use NiFi to call API or using cron Service to schedule it?

2 REPLIES 2

avatar
Community Manager

@Rothmny, Welcome to our community! To help you get the best possible answer, I have tagged our NiFi experts @MattWho @Shelton @SAMSAL who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Master Mentor

@Rothmny 

Based on what you have shared, you should be able to accomplish this with  GenerateFlowFile and ExecuteStreamCommand processors.

The GenerateFlowFile processor would be configured to generate a 0 byte FlowFile and scheduled using Cron so that it creates a FlowFile on your required schedule that will then trigger the execution of the ExecuteStreamCommand processor that will execute your python script.

For calling rest-api endpoints via NiFi dataflows, you could use the InvokeHTTP processor.  Since the InvokeHTTP processor does not require and inbound connection, you could just have it as first processor in a dataflows scheduled with cron as well.  Or, if the invokeHTTP is called after your script successfully executes you could trigger with the FlowFile that exited the the ExecuteStreamCommandProcessor.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt