Support Questions
Find answers, ask questions, and share your expertise

ConsumeAzureEventhub not receiving events after nifi restart

Highlighted

ConsumeAzureEventhub not receiving events after nifi restart

New Contributor

Hi !
I was able to setup ConsumeAzureEventhub, and seems to work fine after a fresh configuration

We have an azure eventhub with 4 partitions. 

The consumer is a nifi 2 node cluster, on which ConsumeAzureEventhub processor is set up to run on "primary node"

After the initial setup with a fresh "Storage Container Name" all seems to work fine. 

All new events are processed, and if I manual start/stop the processor it works ok without repeating events, so connection to storage checkpoint seems to work as expected.

 

However after several days, if nifi is restarted or primary node rotated, sometimes the ConsumeAzureEventhub do not reads new events from the queue.
The only workaround we found is setting up a new container (or delete contents of the existing one) which it solves the initial problem but we get a set of events again (repeated)

 

We are using nifi version 1.11.4

The current configuration looks like the attached screenshot

Screen Shot 2020-12-29 at 11.37.30.png

I would appreciate any help with this setup
Thanks in advance!
 

 

 

1 REPLY 1
Highlighted

Re: ConsumeAzureEventhub not receiving events after nifi restart

Super Guru

check for errors in the logs.

 

stop the processor and make sure it's really stopped.   check the offsets in storagte

 

perhaps the offsets in storage are corrupt or wrong or updated by someone/something else

 

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-azure-nar/1.12.1/org.apache.n...