apache nifi site-to-site "Unable to refresh Remote Group's peers due to null" Error

[#1]

hi there,

I have created a S2S reporting task to the same instance of nifi: 

After enabing it, the port "Bulletin" can get error message however the bulletin board raise warining the message continueously  :

2019-12-06 20:35:13,274 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@7a616d8a Unable to refresh Remote Group's peers due to null

Why and any idea to fix it?

[#2]

@xpelive 

For site-to-site, NiFi attempts to create peers files inside a "state" directory within the NiFi conf directory.
These peers files allow a NiFi instance to share peer information across multiple identical S2S connections (like having multiple Remote Process Groups (RPGs) all configured with same target URL(s)).  This helps reduce overhead associated with every identical instance of S2S retrieving the same details form the same target. It sounds like your S2S connection is working and the peers info is being returned, but it is unable to store that peer information locally. This is why your S2S connection still works even though this peers bulletin is being thrown.

I would suggest you make sure a "state" directory exists within NiFi's conf directory and the NiFi service user (user that owns the NiFi java process) has proper ownership and permissions to navigate that full directory path and read/write to that directory.

Hope this helps you,

Matt

[#3]

@MattWho 

I am running nifi 1.10 in windows10 with admin account and there is the said "state" directory:

here is the full  log about the s2s error:

2019-12-09 20:39:13,124 DEBUG [Timer-Driven Process Thread-10] org.apache.nifi.engine.FlowEngine A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@5a8bd0fa' has been cancelled.
2019-12-09 20:39:13,126 DEBUG [NiFi Web Server-17] org.apache.nifi.web.filter.TimerFilter GET http://localhost:8080/nifi-api/site-to-site/peers from 127.0.0.1 duration for Request ID null: 2 millis
2019-12-09 20:39:13,129 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@f1ae5b8 Unable to refresh Remote Group's peers due to null
2019-12-09 20:39:13,129 WARN [Http Site-to-Site PeerSelector] o.a.n.r.SiteToSiteBulletinReportingTask SiteToSiteBulletinReportingTask[id=ab75497c-016e-1000-c86b-2763551bcb01] org.apache.nifi.remote.client.PeerSelector@f1ae5b8 Unable to refresh Remote Group's peers due to null
2019-12-09 20:39:13,129 DEBUG [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector
java.lang.NullPointerException: null
at org.apache.nifi.remote.client.PeerSelector.persistPeerStatuses(PeerSelector.java:112)
at org.apache.nifi.remote.client.PeerSelector.refreshPeers(PeerSelector.java:306)
at org.apache.nifi.remote.client.http.HttpClient$2.run(HttpClient.java:86)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-12-09 20:39:13,132 DEBUG [Timer-Driven Process Thread-5] o.a.n.controller.tasks.ConnectableTask Triggering StandardPublicPort[id=db33781e-016e-1000-03bb-ceb910fd0a45,name=ErrorIn]

[#4 · Accepted solution]

@xpelive 

There are two state directories.  The one you shared is used by the state-management.xml local state provider.  The state directory I am talking about should have been created inside the NiFi conf directory.

If it does not exist, try creating it manually and make sure your NiFi service user can navigate to it and has proper permission to read and write and delete files within the directory.

Thanks,

Matt

[#5]

@MattWho 

As your advice, I have created the said "state" folder under "conf", however, it raised the same error。

[#6]

@xpelive 

Just want to make sure a couple configurations.

1. The port you are using in the URL (8180) is the "nifi.web.http.port=" and not the "nifi.remote.input.socket.port=" port configured in your nifi.properties file.

2. When you start your NiFi instance, you will see a line output in the nifi-app.log when startup is complete that says the NiFi Ui is available at the following URLs:  Is "localhost:8180" listed in one of those URLs?  If not, you need to use one of the URLs listed in your Remote Process Group.  It might be that localhost is bound to a different NIC on your server than what NiFi is listening on.

3. If you have or will ever have more than 1 NiFi node in a NiFi cluster, using "localhost" is not going to work.  Your NiFi nodes should be configured to use the actual hostnames fo the server on which the service is running.  That hostname should be configured in both these properties:
--- nifi.remote.input.host=
--- nifi.web.http.host=

4. Since your NiFi is not secured, make sure that your nifi.properties file is not configured to be secure:
nifi.remote.input.secure=false.

Null exceptions are particularly difficult to diagnose because a null basically means there was no error handling coded to handle to particular exception.  Thus we need o look at all angles of the configuration to see where the issue may reside.