error nifi connecting as cluster

[#1]

Hi everyone,

I'm setting up nifi to run in cluster, I have 3 instances configured, but when I start them I get this error.

2024-04-10 20:42:59,080 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at myip3:11443; will use this address for sending heartbeat messages
2024-04-10 20:42:59,081 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at myip3/<unresolved>:11443. Will send Cluster Connection Request to this address

2024-04-10 20:42:59,165 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

Has anyone already ever faced this issue?

[#2 · Accepted solution]

@dan_lucas  Still not much info here to be more helpful.   Have you gotten a single nifi node cluster working before trying 3?    Getting the default truststore/keystore setup with the right host certs could be the issue still.   

What version are you using?    These things are much easier in newest versions.

[#3]

@dan_lucas Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi expert @steven-matison  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#4]

@dan_lucas  Not much to go on here, but it seems like you are having some issues resolving the networking for host/hostnames and ports.  Particularly: 

myip3/<unresolved>:11443

You may want to work on providing true host names and ips and trying nifi again.

[#5]

Edit: Hi @steven-matison.

I changed to host ip and also config a trustore.jks and a keystore.jks.

node 2 and 3

127 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at 172.21.139.137/<unresolved>:11443. Will send Cluster Connection Request to this address
2024-04-11 17:49:23,131 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

node 1

INFO [Clustering Tasks Thread-3] o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2024-04-11 17:50:44,334 and sent to 172.21.139.137:11443 at 2024-04-11 17:50:44,443; determining Cluster Coordinator took 2 millis; DNS lookup for coordinator took 0 millis; connecting to coordinator took 1 millis; sending heartbeat took 45 millis; receiving first byte from response took 60 millis; receiving full response took 60 millis; total time was 109 millis
2024-04-11 17:50:46,787 INFO [Heartbeat Monitor Thread-1] o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 1 heartbeats in 11368 nanos
2024-04-11 17:50:47,962 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2024-04-11 17:50:47,962 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2024-04-11 17:50:49,503 INFO [Process Cluster Protocol Request-1] o.a.n.c.p.impl.SocketProtocolListener Finished processing request c83e0c54-a47d-4944-b48d-c32354b016ad (type=HEARTBEAT, length=2343 bytes) from 172.21.139.137:8443 in 54 millis
2024-04-11 17:50:49,504 INFO [Clustering Tasks Thread-3] o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2024-04-11 17:50:49,444 and sent to 172.21.139.137:11443 at 2024-04-11 17:50:49,504; determining Cluster Coordinator took 1 millis; DNS lookup for coordinator took 0 millis; connecting to coordinator took 1 millis; sending heartbeat took 11 millis; receiving first byte from response took 46 millis; receiving full response took 47 millis; total time was 60 millis

[#6 · Accepted solution]

@dan_lucas  Still not much info here to be more helpful.   Have you gotten a single nifi node cluster working before trying 3?    Getting the default truststore/keystore setup with the right host certs could be the issue still.   

What version are you using?    These things are much easier in newest versions.

[#7 · Accepted solution]

Hi Steven,

I'm using nifi version 1.24. I follow your advice and change to hostname now and follow this tutorial to config new trustore and keystore https://blog.opstree.com/2023/11/14/securing-nifi-cluster-with-tls-toolkit/

Everything seems to be working. Thank you.