Created 03-24-2020 04:29 AM
Upgrading to NiFi version - 1.11.3
Issue - SSL Mutual authentication issue
Currently using NiFi - 1.9.0 and implemented SSL mutual authentication and its working fine.
I am trying to upgrade to a newer version and i have installed 1.11.3 version in the parallel mode on the same server where 1.9.0 is installed in different port.
For the newer version i am using the same keystore and truststore along with the client certificates, basically i have just copied these from 1.9.0 version.
when i am trying to access the webgui i am getting
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
But when i run the crul command i am getting the response from the URL
curl --insecure -v --cert-type P12 --cert cert.p12:[passkey] https://nifisitapp01.XXXXX.com:7081/nifi/
output
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [85 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4337 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
{ [1240 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
} [1706 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
} [264 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=XX; ST=XXXX; L=XXXX; O=SOME ORGANIZATION; OU=X OPERATIONAL UNIT; CN=nifisitapp01.XXXXX.com
* start date: Jul 23 06:36:29 2019 GMT
* expire date: Jul 21 06:36:29 2024 GMT
* issuer: DC=COM; DC=XCOMPANY; DC=UAT; CN=XCompany Issuing CA
* SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
} [5 bytes data]
> GET /nifi/ HTTP/1.1
> Host: nifisitapp01.XXXXX.com:7081
> User-Agent: curl/7.47.1
> Accept: */*
>
{ [5 bytes data]
< HTTP/1.1 200 OK
< Date: Tue, 24 Mar 2020 10:27:57 GMT
< X-Frame-Options: SAMEORIGIN
< Content-Security-Policy: frame-ancestors 'self'
< X-XSS-Protection: 1; mode=block
< Strict-Transport-Security: max-age=31540000
< Content-Type: text/html;charset=utf-8
< Vary: Accept-Encoding, User-Agent
< Transfer-Encoding: chunked
< Server: Jetty(9.4.19.v20190610)
<
{ [16037 bytes data]
I am not able to figure out what is actually missing, is it that browser i not passing the right certificate to the server ?
Please share your inputs.
Thank you
Created 03-24-2020 08:15 AM
The "Unable to find valid certificate path to requested target" suggests an incomplete certificate trust chain.
I would suggest looking getting a verbose output for your NiFi's truststore.jks and keystore.jks files.
keystore -v -list -keystore <truststore.jks or keystore.jks>
If this is a NiFi cluster, make sure you check the truststore.jks on all your NiFi nodes, they need to be able to successfully trust each other as well. When you access the UI via any node's URL, that request is sent to the cluster coordinator and replicated to all nodes in your cluster. Those node to node communication with also use mutual TLS authentication.
A complete trust chain will include all Certificate Authority (CA)s (intermediate(s) all the way up to the root CA). When you look at your verbose output for your keystore.jks (it must contain only 1 PrivateKeyEntry that supports both clientAuth and ServerAuth) you will see an owner and issuer for your PrivateKeyEntry. That Issuer is the signer of your certificate. The signer may be an intermediate CA meaning that its not both the issuer and owner its certificate. If that CA has a different issuer, then it was signed by another CA. This means your truststore.jks must also include that CA as well. This process repeats until you reach the root CA. The rootCA will have the same DN for both owner and issuer. Once all those TrustedCertEntries exist in your truststore, you have the complete trust chain.
The same holds true for the client certificate you are using to connect to your NiFi cluster. I noticed it is signed/issued by "localhost". Make sure localhost trustedCertEntry also exists in all your NiFi node's truststore.jks. If the localhost CA was signed by another CA, make sure that you have that traced all the way back to the root CA as well.
Another thing you may want to try to see if it is a node to node SSL issue or a client to node issue, is start only one node in your NiFi cluster and try to access it after it is fully up.
Hope this helps,
Matt
Created on 03-25-2020 06:32 AM - edited 03-25-2020 07:19 AM
@MattWho I tried running the cluster with only one node and it's still giving me the same error
WARN [Replicate Request Thread-1] o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET /nifi-api/flow/current-user to nifisitapp01.xxxxx.com:7081 due to javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
so now its confirmed to be a client to node issue.
I don't understand how come the curl command is working
curl --insecure -v --cert-type P12 --cert cert.p12:[passkey] https://nifisitapp01.XXXXX.com:7081/nifi/
My keystore has server certificate with 3 levels as mentioned below, so its a complete certificate.
Level 1 :
Owner : nifisitapp01
Issuer : Intermediate
Level 2:
Owner : Intermediate
Issuer : Owner
Level 3:
Owner : Owner
Issuer : Owner
The trust store contains all these 3 level certificates and my client certificate and owner(localhost) certificate as well.
Created 03-28-2020 07:28 PM
@MattWho It looks like there are modifications with respect the certificate validation in 1.11.3.
I installed the 1.9.1 version in parallel on different port and i was able to authenticate successfully by using existing keystore and truststore files.
Created 03-31-2020 08:31 AM