question Re: [RESOLVED] : NIFI : LISTENHTTP SSL in Support Questions

[RESOLVED] : NIFI : LISTENHTTP SSL

maykiwogno — Mon, 24 Oct 2016 16:36:35 GMT

Hi all,

I'm trying use listenhttp with ssl.

I've read this post : http://www.simonellistonball.com/technology/nifi-ssl-listenhttp/

But i don't know how use keystore from client. It is the same keystore than keystore used in nifi.properties ?

thanks

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

nlam — Mon, 24 Oct 2016 17:02:13 GMT

Hi @mayki wogno,

If you're talking about a situation where you've got

nifi.security.needClientAuth=True

in your nifi.properties, then for the client, you probably want to generate a separate cert that has been signed by the same CA that your NiFi node(s) trust. I.e. either use the same self-managed CA (such as the one that Ambari installs for you) or use official (mostly paid) CA-signed certificates.

As a proof of concept though, it's certainly fine to re-use the same certs.

Your private key could well be in pkcs12 with a passphrase protecting it, but you should be able to convert it to an unprotected pem file using something like this.

[root@test-nifti conf] # openssl pkcs12 -in keystore.pkcs12 -out keystore.pem -nodes

Then you can test connecting to itself using curl like so.

[root@test-nifti conf]# curl -v --cacert ./keys_test-nifti/nifi-cert.pem --cert ./keystore.pem https://test-nifti.example.com:9443/nifi
* About to connect() to test-nifti.example.com port 9443 (#0)
*   Trying 192.26.217.14...
* Connected to test-nifti.example.com (192.26.217.14) port 9443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: ./keys_test-nifti/nifi-cert.pem
  CApath: none
* NSS: client certificate from file
*       subject: CN=test-nifti.example.com,OU=EXAMPLE.COM
*       start date: Oct 17 23:14:07 2016 GMT
*       expire date: Oct 17 23:14:07 2019 GMT
*       common name: test-nifti.example.com
*       issuer: CN=nifti-ca.example.com,OU=EXAMPLE.COM
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA256
* Server certificate:
*       subject: CN=test-nifti.example.com,OU=EXAMPLE.COM
*       start date: Oct 17 23:14:07 2016 GMT
*       expire date: Oct 17 23:14:07 2019 GMT
*       common name: test-nifti.example.com
*       issuer: CN=nifti-ca.example.com,OU=EXAMPLE.COM
> GET /nifi HTTP/1.1
> User-Agent: curl/7.29.0
> Host: test-nifti.example.com:9443
> Accept: */*
> 
< HTTP/1.1 302 Found
< Date: Wed, 19 Oct 2016 12:33:50 GMT
< Location: https://test-nifti.example.com:9443/nifi/
< Content-Length: 0
< Server: Jetty(9.3.9.v20160517)
< 
* Connection #0 to host test-nifti.example.com left intact
[root@test-nifti conf]#

You can also import that keystore into your browser, so that the NiFi node can authenticate you.

The point of using a separate certificate though, is that you are not sharing the NiFi node(s) “secret” to your clients but instead using the mutual trust of the CA. So make sure you’re using dedicated client certs if you’re doing anything more than basic testing.

Please upvote and accept if this answered your question :).

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

maykiwogno — Mon, 24 Oct 2016 17:44:12 GMT

@Nik Lam : Need some details

In my cluster with 4 nodes, I've used nifi-toolkit for generating keystore/truststore + kerberos for accessing WEB UI. It is OK

In your example, where is come your file keystore?

keystore.pkcs12

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

nlam — Mon, 24 Oct 2016 18:45:10 GMT

It was generated using nifi-toolkit as well with "-T pkcs12".

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

maykiwogno — Sun, 18 Aug 2019 13:20:54 GMT

@Nik Lam : Oh sorry, i don't speak about connecting to web ui

I would use listenhttp with SSL, so in ListenHTTP (Properties ==> use : SSL Context Service)

On the Processor Group Configuration, I need use SSL so how keystore/truststore need to use?

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

MattWho — Mon, 24 Oct 2016 19:45:47 GMT

@mayki wogno

SSL Certificates are generally issued per server. It would be extremely unusual to have multiple certificates issued for the same server or to issue certificates per piece of software. I am not saying you can't, but unnecessary.

Having said that, when you installed NiFi via Ambari, The NIfI CA that was installed likely generated the certificates needed for all your servers where NiFi is running its nodes. A quick look in the nifi.properties file for these jks files will reveal where to find them and the passwords for them as well (Default nifi.properties location on each node --> /etc/nifi/2.0.0.0-579/0/nifi.properties)

Now here is the hitch.... When the NiFi CA generates these keystores for your NiFi nodes, the keystore and truststore on every node end up with its own unique password. This will not work for the ssl context service you need to configure to make your ListenHTTP processor operate using SSL. This is because the same identical ssl context service will be used by the listenHTTP processor on each node.

There is a little work that needs to be done on each node to get you setup here, I suggest you make a copy of the keystore.jks and truststore.jks files on every node in to some new directory. Then capture the following lines from the nifi.properties file on each node:

nifi.security.keyPasswd=<keypass> 
nifi.security.keystore=/<path>/keystore.jks 
nifi.security.keystorePasswd=<keystore-password> 
nifi.security.keystoreType=jks 
nifi.security.truststore=/<path>/truststore.jks 
nifi.security.truststorePasswd=<truststore-password>
nifi.security.truststoreType=jks

Lets assume you create a /opt/nifi-certs/ directory n every node and placed them there.

Next you will want to use the keytool command to change the password on the copies so that nifi.security.keyPasswd= uses the same password on every node, nifi.security.keystorePasswd= uses the same password on every node, and nifi.security.truststorePasswd= uses the same password on every node. While the contents of each certificate is unique to the server, the passwords used to access that key and keystore will then all be the same. This will allow you to configure your ssl-context service to use common passwords to access these server keystores on each node.

Below are the command you can use to change the keystore/trustore passwords and the key passwords within the keystore files.

For each keystore do the following:

keytool -storepasswd -new <new_storepass> -keystore keystore.jks

(Make sure you use the same <new_storepass> on every node in your nifi cluster.

keytool -keypasswd  -alias [Alias name for private key]  -keystore [path to key store]

(If you don't know your private key <alias>, you can get it by running the following command against your keystore.jks file. You will be prompted for yoru <new_storepassword> and the original <key-password>, and then allowed to set a >new-key-password>)

keytool -v --list -keystore keystore.jks

For each Truststore do the following:

keytool -storepasswd -new <new_trustpass> -keystore truststore.jks

(Make sure you use the same <new_trustpass> on every node in your nifi cluster.

After you have completed the above on every node you are ready to configure and start using the ssl-context service.

Thanks,

Matt

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

maykiwogno — Mon, 24 Oct 2016 20:37:55 GMT

@mclark : Oh thanks, I appreciate it.

Just for clear, for each node, I'll copy theirs own keystore/truststore in specific directory and change their passwd with same passwd for all nodes

Need I change something in the client with curl command?

curl -X POST -H 'Content-Type: application/json' -H 'filename: gstat-flume-a1.log.10' -F file=@/var/opt/hosting/log/flume/flume-a1.log.10 http://nifi011:10001/contentListener

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

MattWho — Mon, 24 Oct 2016 21:40:56 GMT

So you created .pkcs12 keystores instead of JKS keystores.

That is fine.... try using the below command to change passwords:

keytool -importkeystore -srckeystore keystore.pkcs12 -srcstoretype PKCS12 -srcstorepass 123456 -destkeystore keystore-new.pkcs12 -deststoretype PKCS12 -deststorepass 11223344

or the following to change it to jks with new password:

keytool -importkeystore -srckeystore <keystore.pkcs12> -srcstoretype pkcs12
 -srcalias <alias> -destkeystore <keystore.jks>
 -deststoretype jks -deststorepass <new-password> -destalias <alias>

Thanks,

Matt

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

MattWho — Mon, 24 Oct 2016 21:50:28 GMT

Your client curl command would not be pointing at a secure https://nifi011:10001/contentListener end-point and will need its own certificate to present in this connection. You can use the toolkit to generate another pkcs12 keystore that your connecting client can use. That way it gets signed by the same CA.

here is some useful information on this topic:

http://callistaenterprise.se/blogg/teknik/2011/04/04/curl-mutual-authentication-and-web-services/

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

maykiwogno — Sun, 18 Aug 2019 13:20:41 GMT

@mclark

I've changed my configuration Controller Service but I'm running an issue

I've checked all truststore/keystore on all the cluster, their use same passwd

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

MattWho — Fri, 28 Oct 2016 18:14:59 GMT

It does not look like you provided you key password.

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

maykiwogno — Mon, 31 Oct 2016 15:34:33 GMT

@mclark

Now that ContextServiceSSL was enabled. I need to generate keystore for my client/user ?

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

maykiwogno — Mon, 31 Oct 2016 16:26:34 GMT

@mclark

I've found that was wrong in my config. Before changing storepass for keystore.jks. I need to change keypasswd for private key first

So Step 1 :

keytool -keypasswd  -alias [Alias name for private key]  -keystore [path to key store]

Step 2 :

keytool -storepasswd -new <new_storepass> -keystore keystore.jks

Now It works

Thanks @mclark

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

MattWho — Mon, 31 Oct 2016 20:03:03 GMT

ListenHTTP requires 2-way SSL when enabled. So the client will also need a keystore and truststore. The Truststore on both your client and server will need to contain the trusted cert entry for each others client cert. If you used the same CA for both then you should be good. If not you will need to add the CA or trusted key entry (Public key from each private key entry.) to each others Truststores.

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

shariquealam786 — Tue, 17 Sep 2019 09:42:28 GMT

Hi Mat,

Sorry to post my question here not getting the way out.

I have configured SSL with CA signed certs on the Server.By CA signed i mean actual CA and not NIFI CA.

I am using Nifi on a single machine without Ambari.

Now i want a client to authenticate and use nifi , how to create the client cert so that it authenticates to the CA signed cert on server.

Re: [RESOLVED] : NIFI : LISTENHTTP SSL

MattWho — Wed, 05 Feb 2020 16:00:39 GMT

Want to add some clarity to this last comment:

ListenHTTP requires 2-way TLS when enabled if a SSLContextService has been configured with a truststore. The truststore is used to trust the client certificate presented by the client, for the purpose of authentication, connecting to this secured ListenHTTP processor.

If only a keystore and no truststore is configured in the SSLContext service, the ListenHTTP will not require that clients present a client certificate.

The server certificate from the keystore will be presented to the client so the client can verify that it trusts the server (NiFI listenHTTP jetty server) that it is connecting with.