Creating a Secure NiFi Cluster with 3rd Party Provided Certificates

davehkd — Sun, 12 Mar 2023 15:29:36 GMT

Hello, I am following the Apache NiFi Walkthroughs, specifically the NiFi Standalone Using External CA. I am trying to install a 5 node cluster. The process I am following is:

I created, using TinyCert the following:

1. A signed NiFi server certifcate for each NiFi server in my cluster i.e., nifi1, nifi2, nifi3, nifi4, and nifi5.

2. The matching private keys in PEM format.

3. A signed client certificate that I created on nifi1 (which I then planned to reuse for the other nodes i.e., nifi2, nifi3, nifi4, and nifi5).

4. The matching private key in PEM format.

5. The CA certificate in PEM format.

Then, on each server i.e., nifi1, nifi2, nifi3, nifi4, and nifi5, i followed the steps indicated in the walkthrough i.e.,

1. concatenate the server certificate and CA certificate to form the certificate chain.

2. Form the PKCS12 keystore from the certificate chain and private key.

3. Convert the PKCS12 keystore for the NiFi instance into the Java Keystore file.

4. Convert the CA certificate into the NiFi trustore

5. Then, for each nifi server, I move the keystore.jks, truststore.jks to the appropriate nifi server.

6. Then, for each nifi server e.g., nifi1, nifi2, etc..., I modified the nifi.properties files for each as indicated

7. I then generated the client certificate keystore from the client certificate and key.

8. For each node, in each respective nifi.properties file, I changed the following:

nifi.cluster.is.node = true, and nifi.cluster.load.balance.host=nifi1 (and nifi2 in the corresponding nifi.properties file for nifi2, and so on...

I then updated each nifi servers nifi.properties file by updating the nifi.zookeeper.connect.string = to the IP address:2181,xxx for each zookeeper (I have 5 zookeeper nodes)

I then updated the state-management.xml file, and added the ip address:2181 of each zookeeper node in the connect string property.

I then updated the authorizers.xml file for each nifi server.

I then started nifi on each nifi server.

Here is the log from nifi-app.log from nifi1:

GNU nano 2.9.8 nifi-app.log

2023-03-12 14:40:31,710 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2023-03-12 14:40:31,769 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loading Application Properties$
2023-03-12 14:40:31,779 INFO [main] org.apache.nifi.NiFi Application Properties loaded [203]
2023-03-12 14:40:31,894 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening$
2023-03-12 14:40:31,908 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication $
2023-03-12 14:40:31,996 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding 128 NAR files with all proces$
2023-03-12 14:45:31,550 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading process took 299554337505 n$
2023-03-12 14:46:17,598 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:17,612 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:22,622 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,019 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,478 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,481 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,534 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,562 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,583 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,594 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:23,603 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,166 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,170 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,205 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,219 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,245 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,256 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,281 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,434 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,441 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,441 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,443 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:24,450 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:25,174 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:26,005 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:27,053 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:27,665 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:27,730 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:27,948 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:28,429 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:28,665 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:28,779 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:29,457 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:30,035 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
2023-03-12 14:46:30,041 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ec2-user/nif$
[ Read 1879 lines ]
^G Get Help ^O Write Out ^W Where Is ^K Cut Text ^J Justify ^C Cur Pos M-U Undo
^X Exit ^R Read File ^\ Replace ^U Uncut Text ^T To Spell ^_ Go To Line M-E Redo

Is this a viable approach for setting up a multi-node secure cluster using 3rd party certificates?

Re: Creating a Secure NiFi Cluster with 3rd Party Provided Certificates

MattWho — Mon, 13 Mar 2023 19:35:52 GMT

@davehkd
I am not sure I am clear on the ask. Are you having issues with your 5 node NiFi cluster?

As far as certificates go for NiFi, it really does not matter where you obtain them or if you use self-signed (not recommended) as long as the keystore meets the requirements for NiFi.

A NiFi node's keystore must meeting the following requirements:
1. Keystore contains only 1 PrivateKey entry. You can not have multiple PrivateKey Entries in the keystore since NiFi will not know which to use.
2. Keystore PrivateKey entry MUST have Extended Key Usage (EKU) of clientAuth and serverAuth, NiFi nodes communicate with one another and thus will act as clients and servers in the TLS exchange.
3. Keystore PrivateKey entry must contain a DNS entry for the hostname on which the certificate is being used.

A NiFi node's truststore contains 1 too many trustedCertEntries. It needs to contain the complete trust chain for any client certificates that will be used to authenticate with NiFi via a mutual TLS handshake. This includes the complete trust chain for each node in yoru cluster. A trust chain consist of every intermediate CA public cert all the way to the root CA public cert. The root CA will have the same owner and issuer. The cacerts file that is included with most java distributions is a truststore containing most public signing authorities intermediate and root CAs.

You can obtain a verbose listing of your keystore/truststore using the keytool command found in yoru java install

<path to JDK>/bin/keytool -v -list -keystore <keystore or truststore filename>

From the output verify following on PrivateKey entry:

(DNSName will have your nodes hostname)

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

Re: Creating a Secure NiFi Cluster with 3rd Party Provided Certificates

davehkd — Mon, 13 Mar 2023 20:43:52 GMT

hello Matt, i was simply trying to understand if my approach for use of 3rd party certificates as I described was an appropriate approach.

Thanks for pointing out the tool. I'll use it.

question Creating a Secure NiFi Cluster with 3rd Party Provided Certificates in Support Questions

Creating a Secure NiFi Cluster with 3rd Party Provided Certificates

Re: Creating a Secure NiFi Cluster with 3rd Party Provided Certificates

Re: Creating a Secure NiFi Cluster with 3rd Party Provided Certificates