@girish6    NiFi processor components are configured to execute based on a run schedule. There are two schedule driven strategies available (Cron Driven and Timer Driven). The Cron Driven scheduling strategy uses a user configured Quartz Cron to set how often the processor will execute.   The Timer Driven scheduling strategy (most common strategy used) uses a user configured run schedule (default run schedule is 0 secs, which means run as often as system will allow).   When a processor executes based on the configured scheduling strategy, it will do one of two things: 1. If the processor has one or more inbound connections, it will check if any of them have any queued FlowFiles.  If none of the connections contain any queued FlowFiles, the processor will yield.  The yield is intended to keep the processors with run schedule of 0 secs from simply constantly requesting CPU threads to check empty inbound connection queues.  No matter the run schedule, a yielded processor will not execute until the yield has expired reducing CPU usage by that processor.   2. Some processor have no inbound connections.  These processors will not yield, but continuously execute on the configured run schedule.  You would not have any such processors in your PG2 since they will have upstream connections to components in PG1.  So for "source" type processors like listSFTP, ListFile, GenerateFlowFIle, or any other processor that does not support an inbound/upstream connection,  if the feed of data is not continuous, it is best to use the Cron Driven scheduling strategy or set a Timer Driven run schedule that is not the default 0 secs to reduce CPU usage. On the face of every processor is a state for Tasks/Time.  The stat tells you how many threads reported as completed in the past 5 minutes and how much cumulative CPU time was used by all those completed threads.  This allows you to see the impact a given processor is having on your CPU. Hope this helps explain cpu usage for you, Matt   ... View more

@girish6    The logic you are trying to accomplish is not possible with the existing scheduling strategies that are available to NiFi processor components? I have no idea which processor you are talking about here, but since you mention it is looking for FlowFiles, I can only assume it is a processor with an incoming connection from another component.   What is your concern with just having it run all the time? What about using a different cron so that it runs every 5 mins during hour 3 each day?  This allows it a full hour each day to look for those incoming FlowFiles. Thanks, Matt ... View more

@girish6    NiFi allows you to specify multiple variable registry files in the nifi.properties file. There is only the one property: nifi.variable.registry.properties   This is a comma-separated list of file location paths for one or more custom property files. These files are only read by NiFi on startup, so any changes or additional files added to this property will not take affect until you restart NiFi.   The variable registry file was added to NiFi before it became possible to set NiFi Variables directly within the NiFi UI on process groups.  The advantage to the newer variables that can be set on process groups is that additional variables and changes to existing variables can be made without needing to restart NiFi.  From the NiFi Ui you can see a list of all components that are currently using a set variable. Changing a variable via the UI also will handle stopping and restarting any component already referencing that variable.   Bonus:  Keep an eye out for "NiFi Parameters" coming soon.  Unlike variables, parameters can be used in any NiFi component property field.  Variables are only support in component properties that support NiFi Expression Language (EL). Parameters will also support sensitive properties (meaning they are able to store passwords in a secure encrypted format.   Hope this helps, Matt   ... View more