I have installed apache nifi with ambari (Versions: Nifi 1.9.0, AMBARI 2.7.3, HDF 126.96.36.199-4) on google cloud. There is an https load balancer in front of a three node instance group. The front end terminates in https, using the standard google supplied let's encrypt certificate. The backend end is https, pointing at the port on the three node instance group where nifi is listening, with ssl enabled. Before enabling ssl, I was able to connect to the http nifi on the backend. With ssl enabled, a tcpdump reveals a Bad Certificate error in the tls handshake between the respective nifi nodes and the load balancer health check IP range.
I am looking for how to configure nifi so that nifi sends the entire chain of certificates and completes a successful handshake with the health check.
Right now my nifi ssl config node entities are: CN=full qualified domain name (FQDN)of node 1, OU=NIFI</property> CN=full qualified domain name (FQDN)of node 2, OU=NIFI</property> CN=full qualified domain name (FQDN)of node 3, OU=NIFI</property> -->
I have created and downloaded certs from TinyCert, and generated the keystore.jks and truststore.jks files which are on each nifi node.
The keystore contains two entries:
Alias name: cert Creation date: Oct 31, 2019 Entry type: trustedCertEntry Owner: CN=FQDN of node 1, OU=NIFI, O=Staq, L=Baltimore, ST=Md, C=US Issuer: CN=Staq CA, OU=Secure Digital Certificate Signing, O=Staq, L=Baltimore, ST=Md, C=US
Alias name: cacert Creation date: Oct 31, 2019 Entry type: trustedCertEntry Owner: CN=Staq CA, OU=Secure Digital Certificate Signing, O=Staq, L=Baltimore, ST=Md, C=US Issuer: CN=Staq CA, OU=Secure Digital Certificate Signing, O=Staq, L=Baltimore, ST=Md, C=US
How do I make the health check successfully make the tls handshake with the nifi node ?
The Google load balancer back end is an instance group with three nodes, nifi1, nifi2, nifi3, where nifi is running on each and listening on port 9091. The backend service is protocol https, named port https, the instance group is entered for the backend and port number is 9091. When I sit on nifi1 and curl the internal ip:9091 of nifi2, in a tcpdump of eth0 activity , I get a fatal error 'Unknown CA' in the handshake from nifi1 to nifi2. In the same tcpdump I see the healch check trying to connect to nifi2 ip:9091, from the range of IP addresses health check uses, and it gets a fatal error during the tls handshake, 'Bad Certificate'. I went to TinyCert.org, and created a certificate for each node, so the CN for each node is the FQDN(fully qualified domain name) , the OU is NIFI, and the rest of the fields are the same. The Certificate Authority CN is Staq CA, the OU is 'Secure Digital Certificate Signing', and the rest of the fields are the same. So I have a cacert.pem , and each node has a set of files: certchain.pfx, cert.pfx, key.dec.pem, key.enc.pem, certchin.pem, and cert.pem. openssl verify -CAfile ../cacert.pem cert.pem cert.pem: OK For NIFI, I generated a keystore.jks and a truststore.jks file, and loaded these to /etc/security/nifi-certs, and put this location in the nifi.properties file. nifi.web.http.host= nifi.web.http.network.interface.default= nifi.web.http.port= nifi.web.https.host=FQDN (same as used in CN name) nifi.web.https.network.interface.default= nifi.web.https.port=9091 The login-identity-providers.xml is empty. I used the following to create the truststore and keystore. openssl pkcs12 -export -in cert.pem -inkey key.enc.pem -out abc.pkcs12 Enter pass phrase for key.enc.pem: PASSPHR Enter Export Password: PASSWD keytool -importkeystore -srckeystore abc.pkcs12 -srcstoretype PKCS12 -destkeystore keystore.jks -deststoretype jks Importing keystore abc.pkcs12 to keystore.jks... Enter destination keystore password: PASSWD Re-enter new password: PASSWD Enter source keystore password: PASSWD Entry for alias 1 successfully imported. keytool -import -file ../cacert.pem -alias cacert -keystore truststore.jks -storepass trust this certficate: yes