question Apache NiFi GetAzureEventHub connection error in Support Questions

Apache NiFi GetAzureEventHub connection error

DataEngineer07 — Mon, 09 Sep 2024 17:05:18 GMT

Hi All,

We have been facing an issue with building a flow in Apache NiFi (version 1.18) to receive events from Azure EventHub.

We had a Process Group that would read events from Azure EventHub using ‘GetAzureEventHub’ processor. It was running successfully, but in March 2024 it stopped working without any changes to NiFi server or on the Azure EventHub. We received the following error - “Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to reactor.core.Exceptions$ReactiveException: java.lang.InterruptedException: reactor.core.Exceptions$ReactiveException: java.lang.InterruptedException - Caused by: java.lang.InterruptedException”

Since then, we have not been able to connect to Azure EventHub from NiFi. We have tried ‘ConsumeAzureEventHub’ processor as well, but received “Connection is disposed. Cannot get management instance” error. Azure EventHub is public, and we’re able to successfully connect to it from outside NiFi (local machine).

While processing some logs, we noticed that GetAzureEventHub processor was sending requests to port 33514 (correct IP address), however, Azure EventHub needs requests in port 5671/5672. As per the NiFi documentation (https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-azure-nar/1.27.0/org.apache.nifi.processors.azure.eventhub.GetAzureEventHub/index.html) as well, default “Transport Type” is “AMQP” which is supposed to send requests to 5671/5672.

So, we’re looking for help to understand 1) why the requests are not being sent to the correct ports? and 2) How can we change the ports, because we do not see any option to change the property “Transport Type”?

Any help or suggestion would be greatly appreciated. Thanks in advance!

Re: Apache NiFi GetAzureEventHub connection error

cjervis — Mon, 09 Sep 2024 20:56:07 GMT

Welcome to the community @DataEngineer07 Perhaps @MattWho or @steven-matison can point you in the right direction.

Re: Apache NiFi GetAzureEventHub connection error

steven-matison — Tue, 10 Sep 2024 12:52:06 GMT

@DataEngineer07 This seems like something external interrupting this process (ie:InterruptedException). I would try a separate nifi instance with updated version to test basic connectivty to Azure Event Hub. That will confirm if its something specific to that original host or nifi version/config. Considering the port difference you notice, i would also investigate that on the affected host as well (proxy or other).

Re: Apache NiFi GetAzureEventHub connection error

cjervis — Mon, 16 Sep 2024 12:05:56 GMT

@DataEngineer07 Have you been able to resolved your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.