Processor not following CRON schedule to execute once at 17:00

Question

I have setup an ExecuteSQLRecord processor to get a list of files that were processed daily at 5pm and send this list via email. The process flow works as expected, except for the CRON schedule. My CRON expression is as follows:

0 0 17 * * ?

I believe this is correct to execute daily at 17:00 (5pm).

However, the process flow executes daily at 16:07 (4:07pm) and 17:00 (5pm).

I am scratching my head trying to figure out why the odd execution at 4:07pm.

Does anyone have any insights to where I can check to see why this is happening?

BobKing · Accepted Answer

We are running the following:Cloudera Flow Management (CFM) 2.1.7.10011.26.0.2.1.7.1001-5 built 07/03/2024 09:38:52 EDTPowered by Apache NiFi 1.26.0 I have 4 CRON driven processors that drive their respective process flows. 3 of the processors have execute 6pm weekdays (0 0 18 ? * MON-FRI) with no issues.  This one executes at 5pm as expected, but also executes at 4:07pm which is strange. This is on a single core in our development environment. We are not using clusters. Also, since this is our development core, only I have access at this time. And I did not manually execute the processor. And I just realized that our old Windows Service process is executing in development and that the 4:07pm emails came from it and not the Nifi CDF process we are developing and testing. So this is resolved and in no way an issue with Cloudera or Nifi.

MattWho · Answer

@BobKing

Unfortunately there is not enough information provided to explain what is going on. At. minimum you should always share the exact NiFi version being used to make it easier to check for any possible known issues affecting your version.

Apache NiFi uses Quartz Cron. The cron you have configured is correct for "scheduling" a processor for execution at 17:00 every day. Keep in mind that once scheduled for execution, how long afterwards it executes depends on thread availability. There are potentially many processors on your NiFi canvas getting scheduled and the "Max Timer Driven Thread Count" pool of threads needs to service all those scheduled processors. Generally thread execution is very fast, but at times you may have your thread pool being used by some longer running threads that could delay execution.

An execution at 16:07 pm seem very unexpected here.

Did maybe someone manually execute the processor using "run once" at that time?
Are you sure the processor ID executed each of those times?
Is this a single instance of NiFi or a Multi-node NiFi cluster setup? If cluster, is the system time synced to a NTP server to make sure all the servers to ensure consistent time across the cluster?

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.