ConsumeImap Processor is not working properly in NiFi

4 replies
2750 views
Translate All

We are using ConsumeImap Processor in NiFi. I have schedule the job to run once in the day. When the job is running at scheduled time its reading all the mails but processing only one.

But when I am triggering it manually, its reading all the mails and processing all the mails.

What could be the issue? below is the configuration for the same

Apache NiFi

MattWho

Titanium ⭐️⭐️⭐️

@Techie123

When you say "run it manually", does that mean you simply start the processor and allow it to run continuously or are you right click on the processor and selecting "run once"?

How do you have the "scheduling" configured for the processor? I assume you are trying to use cron?

Thank you,
Matt

Translate

Techie123Author

Silver ⭐️⭐️⭐️

I am running it once. I am using cron to scheduled the job @MattWho

Translate

MattWho

Titanium ⭐️⭐️⭐️

@Techie123

Different processor are built to execute differently when scheduled to execute by different Apache NiFi contributors.

Some ways to overcome this are to possibly:

Adjust the run duration so that processor runs longer when scheduled to process more FlowFiles or produce more FlowFile from an internal queue in a single scheduled execution.
Configure your cron to execute multiple times within a short period of time. so maybe every second for 15 seconds within a specific hour and minute.
Combination of both the above.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Translate

pawel_3s

Silver ⭐️

I am facing the same issue.

The processor is configured rto get only 10 messages at a time.

I have over 500 messages in the inbox, all unread.

If I "Run once" the ConsumeIMAP processor it get 10 messages every time.

If I "run" the processor (regrdless what is the interval and if it's TimeDriven or CronDriven) it only get 1 message each time, but when I stop the processor it release 9 extra messages . I don't know what happens with the rest of the emails when the process executed multiple intervals, I don't know if anything is lost.

The processor has no configuration for run duration:

My processor configuration:

Translate

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.