Why doesn't my remote groups reconnecting to input port after a Nifi restart.

[#1]

Hi,

I’m trying to implement a Listfile>FetchFile pattern similar to the one described on the following link:

https://pierrevillard.com/2017/02/23/listfetch-pattern-and-remote-process-group-in-apache-nifi/

I have multiple Listfiles process groups each containing a remote process group that point to a common input port.

The input port itself points to a common fetchfile process group.

My issue I’m having is that after I do a Nifi restart not all my remote process groups successfully connect to the input port.

Are there any setting I should be aware to help eliminate this problem? Should use RAW vs HTTP, I’m currently using HTTP.

Any help be greatly appreciated.

Regards,

Bill

[#2]

So at my top level I have my nput port, common fetch process group and multiple list process groups:

I set my input port to support 10 concurrent tasks which I though might help but I’m not really sure.

Each list process group contains the following which includes the remote process group:

I’ve tried both RAW and HTTP, RAW seem to be a little better on recovery be not perfect.

My common fetch process group contains the following:

No real issues here if the remote process group can find the the iinput port everything works as expected.

In my nifi-app.log file I see things like this:

2018-10-23 15:19:46,403 INFO [main] o.apache.nifi.controller.FlowController Started 3 Remote Group Ports transmitting

2018-10-23 15:19:46,403 INFO [main] o.a.n.w.c.ApplicationStartupContextListener Flow Controller started successfully.

2018-10-23 15:19:46,650 WARN [Remote Process Group 763b0492-57ec-3bdb-71c0-b6c607183d9c Thread-1] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from http://localhost:6080/nifi-api due to org.apache.http.conn.HttpHostConnectException: Connect to localhost:6080 [localhost/127.0.0.1] failed: Connection refused: connect

2018-10-23 15:19:46,653 WARN [Remote Process Group 763b0492-57ec-3bdb-71c0-b6c607183d9c Thread-1] o.a.n.remote.StandardRemoteProcessGroup Unable to communicate with remote instance RemoteProcessGroup[http://localhost:6080/nifi]

org.apache.nifi.controller.exception.CommunicationsException: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI http://localhost:6080/nifi due to: Connect to localhost:6080 [localhost/127.0.0.1] failed: Connection refused: connect

at org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:883)

at org.apache.nifi.remote.StandardRemoteProcessGroup.lambda$initialize$0(StandardRemoteProcessGroup.java:193)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI http://localhost:6080/nifi due to: Connect to localhost:6080 [localhost/127.0.0.1] failed: Connection refused: connect

at org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:845)

... 8 common frames omitted

2018-10-23 15:19:47,664 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from http://localhost:6080/nifi-api due to org.apache.http.conn.HttpHostConnectException: Connect to localhost:6080 [localhost/127.0.0.1] failed: Connection refused: connect

2018-10-23 15:19:47,665 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@31c739e8 Unable to refresh Remote Group's peers due to Connect to localhost:6080 [localhost/127.0.0.1] failed: Connection refused: connect

2018-10-23 15:19:48,654 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@2c952e6{/nifi-api,file:///C:/Dev/aim_project_repos/nifi-1.7.1/work/jetty/nifi-...

2018-10-23 15:19:48,668 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from http://localhost:6080/nifi-api due to org.apache.http.conn.HttpHostConnectException: Connect to localhost:6080 [localhost/127.0.0.1] failed: Connection refused: connect

2018-10-23 15:19:48,668 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@750d934c Unable to refresh Remote Group's peers due to Connect to localhost:6080 [localhost/127.0.0.1] failed: Connection refused: connect

Restarting sometimes clears the issue.

[#3]

@Willian Gosse

-

During a NiFi restart, the flow is loaded and started before the NiFi UI is made available. During this period of time the Remote Process Groups (RPG) on each node will fail to be able to connect to the configured target NiFi URL to fetch the Site-To-Site (S2S) details. This is expected behavior. The RPGs will stop throwing this error in the logs once the configured target NiFi URL is made available and the S2S details are successfully retrieved.

-

The use of HTTP or RAW as the transport protocol controls how the actual FlowFiles are transferred. The re-occurring connection to retrieve the S2S details will always be over http to the target NiFi URL configured in the RPG.

When using HTTP transport protocol. the NiFi FlowFiles will also be transferred via the same HTTP port as the Target NiFi UI is exposed on.

Setting transport protocol to use RAW causes the RPG to use a dedicated socket port for the FlowFile transfer. The socket port used is set by the target NiFi servers in the nifi.properties file (property: nifi.remote.input.socket.port=).

The advantage to using RAW is that amount of traffic going to HTTP port used to access UI is reduced considerably.
The advantage to using HTTP is that you have one less port you must open through any firewalls to the NiFi nodes.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.