Nifi - How to enable mutual certificate authentication between Nifi (Client) and REST API (Server)

[#1]

I am invoking a REST API from Nifi using InvokeHTTP processor and wanted to enable mutual certificate authentication between client (nifi) and server.

I enabled StandardRestrictedSSLContextService (controller service) as mentioned in the below screenshot and getting below exception:

InvokeHTTP[id=4a29f282-3665-3703-aab8-9e106c4aa9a4] Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to java.lang.reflect.InvocationTargetException: java.lang.reflect.InvocationTargetException

Screenshot:

Am I missing anything, kindly please explain how to enable mutual certificate based authentication?

Cc: @MattWho , @arunak 

[#2]

@VijaySankar In a recent Use Case, I had to connect invokeHttp to an API (https) to get an S3 bucket URL and then to Amazon S3 (https) to download the file.  Each InvokeHTTP required its own SSL Context Service and for each I complete the following:

1. Obtain the public SSL certificate from the end point.  I use curl, or standard browser to download the cert.
2. Use the cert to make a truststore (JKS) file.
3. Put this truststore on all my NiFi nodes, owned nifi:nifi
4. Put the trustore path, type, and password in the SSL Context Service.  
5. Enable Context Service, and test InvokeHTTP

Hope this helps get you going...

[#3]

Thanks @stevenmatison

As you mentioned, I have configured the InvokeHttp processor and sent a request to server and received the response.

Is there any way to verify the outgoing request has the certificate attached from Nifi side (client)?

[#4]

If it worked and you got the expected response, you can be sure the cert was used... i don’t think the cert will actually be “sent” but it is used on the Nifi side (client) to communicate “securely”...  just make sure you get the expected response and not, for example, an http (not https) error page...

[#5]

@VijaySankar 

Whether a client certificate is required in the TLS connection is controlled by the server side of teh connection and not by the client.

Your SSL context service must include a truststore.  The truststore is required whether the TLS secured connection is using mutual authentication or 1-way TLS authentication.

The client initiates a connection to the https server.  The server responds with a TLS server hello response. Among other things, the server will include its server certificate, whether a client certificate is required (client certificate must be returned), wanted (client certificate is wanted but connection will proceed even if one is not provided), or none (no client certificate requested), and a list of server side trusted CAs.  If the server does require or want a client certificate, the client can only pass a certificate that is signed (issuer)  is one of the trusted CAs returned in the Server Hello.  If your server endpoint requires a client certificate and non is provided, the server would just close the connection since the TLS handshake failed.

The truststore used in your SSL context service on the client side must be able to trust the server certificate presented (true for mutual auth TLS or 1-way TLS).  This means the truststore you use must contain the complete trust chain for the issuer of the server certificate sent by the server.

If you want to see the server hello, you can use openssl as follows:

openssl s_client -connect <server hostname>:<server port>

If you want to get the public certs for the complete trust chain from the server:

openssl s_client -connect <server hostname>:<server port> -showcerts

You can copy each public cert into a cer file and import those it to your truststore used in the SSL context service if missing.

Hope this helps,

Matt

[#6]

As mentioned in the below screenshot, I have added client cert (.jks) in keystore and server cert (.jks) in truststore. Server is expecting a client cert as part of request but when I make a rest call, client certificate is going in the request.

Here is the command that I used to generate a .jks file on top of client certificate:

keytool -importkeystore -srckeystore nifi.com.pfx -srcstoretype pkcs12 -destkeystore nifi.com.jks -deststoretype jks -srckeypass 'abcxyz' -alias data-nifi-cd

 

@MattWho 

[#7]

@VijaySankar 

Unfortunately with the information you have shared, it is not possible for me to validate what the issue is here.

I would need to output from the openssl commands i shared earlier to help.

openssl s_client -connect <server hostname>:<server port>

Also the verbose listing of both your keystore and truststore files would be needed:

keytool -v -list -keystore keystore.jks

keytool -v -list -keystore nifi.com.jks

The target server is what decides if connection is going to use mutual auth or not.  The invokeHTTP is the client side of this connection and only provides what the server asks for in the server hello response.  
It is possible also in that server hello response that the server wants a client certificate; however, the list of CAs sent in that server hello response does not contain a CA capable of trusting your clientAuth PrivateKeyEntry in your nifi.com.jks.  In that case client would not send the client certificate since server is incapable of trusting it.

Matt