Why ConsumeAzureEventhub processor consumes more data even though output connection is configured with less ?

[#1]

Hello Experts,

We have couple of Eventhub consumers running in nifi 1.16.3 version.

The output connection is configured with default backpressure - 10,000 messages and 1GB.

But I see 'ConsumeAzureEventhub' is keep on consuming data even after crossing backpressure.

What is the reason for this behavior and how to fix this?

@mattw 

Thanks

Mahendra

Thanks

Mahendra

[#2]

@hegdemahendra 

The FlowFile connection back pressure thresholds are soft limits.  Once one of the configured back pressure thresholds on reached or exceeded, NiFi will not allow the processor feeding that connection to get scheduled to execute again.

So in your case no back pressure is being applied, so the ConsumeAzureEventHub processor is being allowed to scheduled to execute.  During a single execution it is consuming more events then the threshold settings.  
What is the batch size set to in your ConsumeAzureEventHub processor?

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt