Apache Nifi PutFile Processor

This can be done with CRON if option is available, otherwise you can automate the process using python and nipyapi.

@Ankit13 

NiFi processors support "Timer Driven" and "Cron Driven" Scheduling Strategies.

• Timer driven strategy allows you to specify a time interval for scheduling (for example: "30 secs" which means processor will execute every 30 seconds.).
• Cron Driven strategy supports a Quartz Cron [1] being used to specify when the processor should execute.

There is a third option on some processors which is Event Driven that should not be used. It was created long ago and considered experimental.  It is has since been deprecated due to improvement made in the Timer Driven strategy.  It only remains in NiFi to avoid breaking flows of those who use it when they upgrade.

1. Important things to understand about your ask:
   Let's assume you configure your PutFile to execute using the Cron Driven scheduling strategy and the inbound connection to the putFile processor has multiple FlowFiles queued.  When the processor executes it will process only 1 of those FlowFiles from that inbound connection queue with default settings.  The next queued FlowFile would not get processed until the next scheduled cron execution.  While there is no way to make sure that every queued FlowFile is processed in in a single cron execution you can change the configured Run Duration:The Run Duration tells the processor to continue to use the same execution thread to execute against as many queued FlowFile as possible within the configured run duration time. Let say it takes more than 2 secs to write the very first FlowFile to the target directory.  In that case, only one FlowFile would be processed.  So there would be no perceived difference between a run duration of 0ms and 2s.
2. In a NiFi cluster, each node in the cluster executes the dataflow against the FlowFiles queued on that same node.  So if FlowFiles were queued on the inbound connection to PutFile on all nodes, each node would execute 1 at each cron interval processing through FlowFile(s) per node as described above.

[1] https://community.cloudera.com/t5/forums/replypage/board-id/Questions/message-id/229905

If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt