Created 12-29-2020 06:43 AM
Hi !
I was able to setup ConsumeAzureEventhub, and seems to work fine after a fresh configuration
We have an azure eventhub with 4 partitions.
The consumer is a nifi 2 node cluster, on which ConsumeAzureEventhub processor is set up to run on "primary node"
After the initial setup with a fresh "Storage Container Name" all seems to work fine.
All new events are processed, and if I manual start/stop the processor it works ok without repeating events, so connection to storage checkpoint seems to work as expected.
However after several days, if nifi is restarted or primary node rotated, sometimes the ConsumeAzureEventhub do not reads new events from the queue.
The only workaround we found is setting up a new container (or delete contents of the existing one) which it solves the initial problem but we get a set of events again (repeated)
We are using nifi version 1.11.4
The current configuration looks like the attached screenshot
Created 01-05-2021 01:31 PM
check for errors in the logs.
stop the processor and make sure it's really stopped. check the offsets in storagte
perhaps the offsets in storage are corrupt or wrong or updated by someone/something else
Created 03-12-2021 09:53 AM
Hi, did you solved this issue? we are facing the same problem and not found any solution.
I apprecite any help, thanks!
Jorge
Created 03-27-2021 11:50 AM
No yet. We decided to move out ConsumeAzureEventhub because of this issue
Created 04-01-2021 03:51 PM
Move all events to Kafka