Support Questions
Find answers, ask questions, and share your expertise

Nifi Standalone to Nifi Cluster Secure Setup issues

New Contributor

I'm having trouble getting an S2S setup working between a standalone Nifi instance and a 6-node Nifi cluster. The intention is data picked up by the standalone will be pushed through to the cluster via an RPG and input port.

I believe I've done the usual things; I've added each side's certs to each side's truststores (standalone got all 6 from the cluster although I don't think it was necessary), all cluster nodes have the cert from the standalone.

Appropriate users created and policies added to allow remote site-to-site.

When I test these procedures on two lab standalone instances all works as expected. Currently I am seeing this error on my cluster:

2018-07-16 15:45:40,475 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@5d46e94b Unable to refresh Remote Group's peers due to Unable to communicate with remote NiFi cluster in order to determine which nodes exist in the remote cluster

More info from the log files:

2018-07-16 15:45:40,474 ERROR [NiFi Site-to-Site Connection Pool Maintenance] o.a.n.r.io.socket.ssl.SSLSocketChannel org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel@638bd302 Failed to connect due to {}
javax.net.ssl.SSLHandshakeException: Reached End-of-File marker while performing handshake
at org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performHandshake(SSLSocketChannel.java:248)
at org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.connect(SSLSocketChannel.java:163)
at org.apache.nifi.remote.client.socket.EndpointConnectionPool.establishSiteToSiteConnection(EndpointConnectionPool.java:455)
at org.apache.nifi.remote.client.socket.EndpointConnectionPool.fetchRemotePeerStatuses(EndpointConnectionPool.java:389)
at org.apache.nifi.remote.client.PeerSelector.fetchRemotePeerStatuses(PeerSelector.java:385)
at org.apache.nifi.remote.client.PeerSelector.refreshPeers(PeerSelector.java:352)
at org.apache.nifi.remote.client.socket.EndpointConnectionPool$2.run(EndpointConnectionPool.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-07-16 15:45:40,475 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector Could not communicate with node01.xyz.com:10000 to determine which nodes exist in the remote NiFi cluster, due to javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
2018-07-16 15:45:40,475 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@5d46e94b Unable to refresh Remote Group's peers due to Unable to communicate with remote NiFi cluster in order to determine which nodes exist in the remote cluster

I'm kind of at a loss as to how to troubleshoot further.

1 REPLY 1

Master Guru

@Chris Murray

-

This is purely a SSL Handshake exception which really is not an issue in NiFi. This is an issue with your keystore and/or truststore contents.

-

Troubleshooting this requires you to inspect the contents of the keystore and truststore being used on both sides of this connection.

This includes inspecting all of your keystores to make sure:

1. Each contains only a single PrivateKeyEntry
2. That PrivateKeyEntry has an extended key usage that supports both ClientAuth and ServerAuth
3. The PrivateKeyEntry will have a "Owner" and "Issuer". Verify that the truststore.jks file on all NiFi instances contains a TrustedCertEntry that matches the "issuer" for the PrivateKeyEntry found in each unique nifi instance keystore.

4. The truststore.jks file can contain 1 to many unique trustedCertEntry.

-

The following command can be used to get the verbose keystore/truststore output:

keytool -v -list -keystore <keystore or truststore file>

-

Thank you,

Matt

-

If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer.