Member since
04-03-2018
5
Posts
0
Kudos Received
0
Solutions
12-04-2018
01:59 PM
@Faheem Shoukat What I am seeing in your screenshot looks as expected. A processor that is started and uses the Timer Driven Scheduling strategy will execute based upon the configured run schedule. The default run schedule is always 0 secs (which means execute as fast as possible). To prevent a processor that has no work to do from consuming 100% of a cpu looking for work non stop, there is a configurable yield duration (default is 1 sec). Based on your screenshot I am seeing ~293 tasks being executed every 5 minutes. This is ~1 task a second which aligns with the design explained above where a yield of 1 sec is being applied between each task. - As far as how busy this keeps a CPU, for these processors that had no work at all to do for the entire 5 minutes, your screen shots shows a cumulative CPU time for all 293 tasks of only ~0.006 seconds (this is not CPU time for each task. It is the cumulative total CPU time reported by all tasks marked as completed within that 5 minute window.) - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.
... View more
04-03-2018
10:51 AM
@Faheem Shoukat Additional Where Clause property has been introduces from NiFi 1.4.0+ versions of NiFi, As you are using NiFi 1.3.0 that's the reason why you are not able to find the Additional where clause property in your QuerydatabaseTable processor. Possible ways to achieve where clause functionality:- You can upgrade your NiFi instance to newer versions (or) Use Execute Sql processor(add where clause in your query), store the state in hive/hbase and pull the state again for incremental run using Execute Sql processor (or) Use combination of GenerateTableFetch(supports additional Where clause property) + RemoteProcessorGroup + ExecuteSql processors to achieve where clause property. NiFi 1.4 Querydatabase table processor configs:- once you have nifi 1.4 version then click on configure on QueryDatabase table processor you are going to have Additional Where clause property, Below is the reference link for Jira ticket addressing about additional where clause property. https://issues.apache.org/jira/browse/NIFI-4257
... View more