Helping setting up cron-based nifi processor

[#1]

I'm trying to setup a nifi processor to run once daily, using the 'cron' option under scheduling.

Under "run schedule", I put "01 18 * * * ?" , which should be 6:01pm (I couldn't get nifi to accept it without the question mark).

But this causes the scheduler to run on the hour, every hour.

Can someone please help me with my syntax here?

[#2 · Accepted solution]

@Zack Riesland

The Cron you have there should run 18 minutes and 1 second into every hour.

What you really are looking for as a cron here for 6:01pm is ( 0 1 18 * * ? * or 0 1 18 * * ? )

• CRON driven: When using the CRON driven scheduling mode, the Processor is scheduled to run periodically, similar to the Timer driven scheduling mode. However, the CRON driven mode provides significantly more flexibility at the expense of increasing the complexity of the configuration. This value is made up of seven fields (where the seventh field is optional), each separated by a space. These fields include:
  • Seconds
  • Minutes
  • Hours
  • Day of Month
  • Month
  • Day of Week
  • Year

The value for each of these fields should be a number, range, or increment. Range here refers to a syntax of <number>-<number>. For example,the Seconds field could be set to 0-30, meaning that the Processor should only be scheduled if the time is 0 to 30 seconds after the minute. Additionally, a value of * indicates that all values are valid for this field. Multiple values can also be entered using a , as a separator: 0,5,10,15,30. An increment is written as <start value>/<increment>. For example, settings a value of 0/10 for the seconds fields means that valid values are 0, 10, 20, 30, 40, and 50. However, if we change this to 5/10, valid values become 5, 15, 25, 35, 45, and 55.

For the Month field, valid values are 1 (January) through 12 (December).

For the Day of Week field, valid values are 1 (Sunday) through 7 (Saturday). Additionally, a value of L may be appended to one of these values to indicate the last occurrence of this day in the month. For example, 1L can be used to indicate the last Monday of the month.

Thanks,

Matt

[#3]

NiFi does not use a linux cron. It uses a Quartz cron/scheduler.

[#4]

When I enter "0 0 18 * * * ?", I get this error message: "Scheduling Period '0 0 18 * * * ?' is not a valid cron expression: '?' can only be specified for Day-of-Month or Day-of-Week"

However, "0 0 18 * * ? *" seems to work.

What is the difference between the meaning of "*" and "?" ?

[#5]

@Zack Riesland

There are seven fields; however, the seventh field is optional. So you are correct.

so both " 0 0 18 * * ? " and " 0 0 18 * * ? * " are valid.

The below is from http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html

----------------

• * (“all values”) - used to select all values within a field. For example, “” in the minute field means *“every minute”.
• ? (“no specific value”) - useful when you need to specify something in one of the two fields in which the character is allowed, but not the other. For example, if I want my trigger to fire on a particular day of the month (say, the 10th), but don’t care what day of the week that happens to be, I would put “10” in the day-of-month field, and “?” in the day-of-week field. See the examples below for clarification.

-----------------

so only fields 4 and 6 will accept ?.

Thanks,

Matt

[#6]

Thanks Matt,

So so would "0 0 18 * * ?" indicate that the job should be run 18 hours into every day?

[#7]

The Quartz scheduler has 7 fields, so the cron would need to be 0 0 18 * * ? *. The seventh field is optional for year. Yes the cron you have there will run the 18th hour of every day.

[#8]

Hey @Matt,

I am using a cron job scheduling strategy too and my cron job is "0 30 13 * * ?" similar to what is discussed here. But, for some reason, it just runs for the day I create it and doesn't repeat the next day. Do you know the reason why? I keep NiFi running and don't think it's shutting down for the cron job to not execute. Any help appreciated.

[#9]

@raghav130593 

Where "*" indicates every value, the "?" can mean i don't care about the value.

So you want to have corn execute at 17:30:00 every day, So i would recommend a "*" instead of "?" in the day of week field.

A good example of when to use a ? in a supporting field would be this cron:
0 30 17 5 * ?
The above says execute at 17:30:00 on the 5th of every month.  The day of week field here is a "?" mark because the 5th of every month could fall on any one of the days of the week (will vary from month to month).

Thanks,

Matt

[#10]

Below is the syntax I have tried with 0 30 0 * * ? or 0 30 1 * * ?, which should run at 12:30 or 1:30 midnight respectively.

Also I have tired running the flow with cron in nifi for every 20 minutes duration, it worked perfectly.

But we want it to be run, daily once at particular time as mentioned above example (0 30 0 * * ? or 0 30 1 * * ?).

But it is not working. can anyone please guide.

Thanks.

[#11]

Superb. Thanks for this great information. the 7 fields is something that I wasn't aware of . drilling down at a second level is great ! @Matt Clarke great answer