Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Monitor Activity Processor Not generating any flow file

Solved Go to solution
Highlighted

Monitor Activity Processor Not generating any flow file

Hello Guys,

I have used Query Databases processor for extracting delta record from table but in cases if delta is not there Querydatabases processor will not generate flow file and my down stream job depends on flow file .So i have used monitor activity and designed a flow and when was the first run its working fine in cases of Zero delta record and it has generated flow file but now in day 2 same situation happened and in table there was not delta record and now monitor activity processor is not generating flow file.

Can any one suggest me what can be a problem?

Query data bases processor i have given a CRON Driven at 9 am UTC and Monitor Activity at 10 am UTC with Thresold Value 5 min.

Regards,

Shantanu

flow-nifi.png flow-nifi-2.png

flow-nifi.pngflow-nifi-2.png
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Monitor Activity Processor Not generating any flow file

Super Guru

@Shantanu kumar

You have to run Monitor Activity processor until 10:05am UTC because threshold value is set to 5 min, if you run monitor activity processor at 10 am then processor run at 10:00am only not at 10:01am, so there is no window has been created for 5 min for monitor activity processor to know is there any flow files for last 5 min (or) not.

Change the schedule of Monitor Activity processor to 0 0-5 10 1/1 * ? *

now processor runs at 10:00 am,10:01am,10:02am,10:03am,10:04am after 10:04am processor has been ran for 5 mins,now at

10:05am run processor sends an inactivity flow file and triggers your ExecuteSql Processor.

View solution in original post

13 REPLIES 13
Highlighted

Re: Monitor Activity Processor Not generating any flow file

Super Guru

@Shantanu kumar

You have to run Monitor Activity processor until 10:05am UTC because threshold value is set to 5 min, if you run monitor activity processor at 10 am then processor run at 10:00am only not at 10:01am, so there is no window has been created for 5 min for monitor activity processor to know is there any flow files for last 5 min (or) not.

Change the schedule of Monitor Activity processor to 0 0-5 10 1/1 * ? *

now processor runs at 10:00 am,10:01am,10:02am,10:03am,10:04am after 10:04am processor has been ran for 5 mins,now at

10:05am run processor sends an inactivity flow file and triggers your ExecuteSql Processor.

View solution in original post

Highlighted

Re: Monitor Activity Processor Not generating any flow file

Thank you Shu,

If i will change the threshold value 5 min to 1 min then in this cases i need to change scheduling of Monitor Activity processor?

Regards,

Shantanu

Highlighted

Re: Monitor Activity Processor Not generating any flow file

Super Guru
@Shantanu kumar

Yes, if you change threshold value to 1 min then we need to run Monitor Activity processor for 2 mins i.e 0 0-1 10 1/1 * ? *, for the first minute checks flow's activity for 1 minute and second minute(10:01am) to trigger a inactivity flowfile.

Now processor will run at 10:00am with this configuration window will be 1 min and checks if there are any flowfiles and 10:01 am(sends an flowfile for inactivity relation if there are no flow files).

Highlighted

Re: Monitor Activity Processor Not generating any flow file

Thanks Shu.

what is 1/1 in your corn scheduling ? will it impact for next run if i will schedule on 0 0-1 10 1/1 * ? *,


Regards,

Shnatanu

Highlighted

Re: Monitor Activity Processor Not generating any flow file

Hello Shu.

Sorry to bother you again.just I am curious for above Cron schedule 0 0-1 10 1/1 * ? *,will work on daily base or only for one day because my jobs scheduled to run daily.could you please give me a reply on it

Regards

Shantanu

Highlighted

Re: Monitor Activity Processor Not generating any flow file

Super Guru
@Shantanu kumar

1/1 is nothing but daily it is similar to * i.e 0 0-1 10 * * ? * (or) 0 0-1 10 1/1 * ? *.

It runs daily at 10:00,10:01 am, you can create/check all cron tab expressions in the following link.

http://www.cronmaker.com/

For more details about cron expression

https://community.hortonworks.com/questions/63513/helping-setting-up-cron-based-nifi-processor.html

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html

Highlighted

Re: Monitor Activity Processor Not generating any flow file

Thanks Shu,

I have used above Cron scheduled but today the behaviour of Monitor activity is very strange, yesterday it ran successfully when in QueryData base processor was not getting any delta record it has generated flow file and sent to execute sql but today again there was not data available for query databases but today Monitor activity not triggered. I do not know what I am missing here flow is same as above just in scheduling i have given 0 0-1 10 1/1 * ? * with threshold value 1 min. Could you please suggest here .I am attaching flow setting for Monitory activity. I have given Execution on Promary node is there any thing with this and Automatically Terminate Relationships i have given activity.restored and success while connecting to QueryDatabases and inactive while connecting to Execute Sql. Please have a look in below screen shot for more detail

64854-monitor-cron-node.png

64855-monitor-flow-file.png

Regards,

Shantanu.

Highlighted

Re: Monitor Activity Processor Not generating any flow file

Another thing wanted to add here .For copy Attributes we need to set True i have given false .You can look in to below screen shot.

64857-monitor-copy-attribt.png

Regards,

Shantanu

Highlighted

Re: Monitor Activity Processor Not generating any flow file

If i will set Continually Send Messages is true then everyday if data will not available then i can get flow file? right now it has in false stat.

Don't have an account?
Coming from Hortonworks? Activate your account here