The purpose of this article is to provide the steps
needed to create your own certificates for securing your NiFi instance(s). The
article will also cover creating your own Certificate Authority (CA) that you
can use to sign all the certificates you create. This article is not intended to be a best practices guide to creating secure keys. While we will provides tips, users should carefully research the various security options available when creating keys. This procedure assumes you have Java Keytool
and OpenSSL installed on your system.
HDF 1.x or Apache NiFi 0.x Secured UI:
HDF 2.x or Apache NiFi 1.x Secured UI:
Creating your Certificate Authority:
You only need to create one CA, which you will use to sign the keys for every one of your servers/VMs and users (You only need to create keys for users if your NiFi has not been configured to use LDAP authentication). What is a CA? The CA acts as a trusted entity for validating the authenticity of certificates. The CA is used to certify the authenticity of the keys (server and user) you create and should be carefully protected. User should read the following wiki on CAs for a more detailed description: https://en.wikipedia.org/wiki/Certificate_authority
Commands for creating a CA:
*** Users should use strong passwords whenever prompted. When working with Java keystores, it is recommended that both the key password and the keystore password match.
*** NOTE: Security requirements are more stringent as newer versions of browsers and NiFi are being used since this article was originally written. The below command should be changed to use "-aes256".
***You must type 'yes' to trust this certificate.
The following command can be used to do a verbose listing of the contents of the above created keystore:
keytool -v -list -keystore truststore.jks
At the end of the above you will have your "truststore" file (truststore.jks) that you will use in your nifi.properties file. Use this same "truststore" file on every one of your servers/VMs. You may also choose to load the rootCA.der or rootCA.pem key into your browser as another authority. This is not required, but without this authority loaded you will need to add a certificate exception when you try to access the NiFi https URL.
Edit the following lines in your nifi.properties file:
nifi.security.truststore=/<path to certs>/truststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=<MyTruststorePassord>
nifi.security.needClientAuth=true
Creating your Server Keystore:
Now lets create a server/vm key and get it signed by that CA:
*** Users should use strong passwords whenever prompted. When working with Java keystores, it is recommended that both the key password and the keystore password match.
The following procedure will [1] create your server/VMs private key, [2] Generate a Certificate Signing Request (.csr), [3] Use CSR to get your key signed by your CA using the CAs private key, [4] Import the public key for your CA in to your keystore, and [5] Import your signed certificate (.crt) in to your keystore to form the complete trusted chain.
At the end of the above you will have your "keystore" file (nifi-server1.jks) that you will use in your nifi.properties file for one of your servers/VMs. You will need to repeat the above steps for each of your other servers/VMs so they each use their own keys. Now keep in mind that I am using “nifi-server1" in this example, but you will most likely use your systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly recommend that you use the same key and keystore password for every key you create if creating keys for multiple nodes in a NiFi cluster.
The following lines need to be edited in the nifi.properties file:
nifi.security.keystore=/<path to your certs>/nifi-server1.jks
nifi.security.keystoreType=JKS
nifi.security.keystorePasswd=<yourkeystorePassword>
nifi.security.keyPasswd=<yourKeyPassword>
Also make sure that you set the following property in the nifi.properties file to true:
nifi.security.needClientAuth=true
Additional configurations for NiFi clusters only:
When working with a NiFi cluster, it is recommended that you change the default NiFi user authority provider. The default is file-provider. On your NCM you should change file-provider to cluster-ncm-provider and on your nodes file-provider should be changed to cluster-node-provider.
nifi.security.user.authority.provider=
You will also need to edit the authority-providers.xml file to configure both of these new providers. Remove the comments ( “<!--“ and “-->” ) surrounding the section of XML associated to the provider you are enabling:
Example NCM provider configuration:
Example Node provider configuration:
Creating User Keys for key based authentication:
Now that you have all the keys you need for the systems in your cluster, you will need to create some keys for your users to load into their web browsers in order to securely access your NiFi. This step is not necessary if you have setup your NiFi to use LDAP for user authentication. This is done in much of the same way as you created your server keys:
*** Users should use strong passwords whenever prompted.
Now you have a p12 file for user1, they can load this in to their browser certs to use to authenticate against your secure NiFi. Import your <user1>.p12 file in to your certificates for your preferred browser.
---------
HDF 1.x or Apache NIFi 0.x only: Now remember you must manually add that first "ROLE_ADMIN" user to the authorized-users.xml file. So you will need the DN from the user key you created for this Admin user and add it in to your Authorized-users.xml file.
---------
HDF 2.x or Apache NiFi 1.x only: You must configure your "Initial Admin Identity" in the authorizers.xml file. That Initial Admin Identity value must match the user's DN from the .p12 file exactly.
---------
Here is an example of what it may look like:
dn="EMAILADDRESS=none@none.com, CN=<user1>, OU=NiFi, O=someplace, L=Baltimore, ST=Maryland, C=US"
Troubleshooting authentication issues:
If you have the DN format wrong in your authorized-users.xml file, rather then gaining access to the NiFi you will get prompted to "request access”. Do not click the request access link. You must instead go fix the DN in the authorized-users.xml file. You need to create that first admin account that can approve those requests. If you click request access, you will need to stop your NiFi and delete the nifi-users.h2.db file (located inside the database_repository directory), otherwise, even fixing your authorized-usesr.xml file will not gain you access because your account will be stuck in a pending auth state.
You can look at the request that came in in the nifi-users.log to get the exact DN pattern to fix your authorized-usesr.xml file entry:
You should see something that looks like this:
INFO [NiFi Web Server-58023] o.a.n.w.s.x509.X509AuthenticationFilter Attempting request for (<CN=JohnDoe, OU=MyBusiness, O=MyOrg, L=Baltimore, ST=MD, C=US>) GET...
That log line gives you the exact format of the DN that needs to be updated/added to the authorized-users.xml file. Example below:
<user dn="CN=John Doe, OU=MyBusiness, O=MyOrg, L=Baltimore, ST=MD, C=US"> <role name="ROLE_DFM"/> <role name="ROLE_ADMIN"/> <role name="ROLE_PROVENANCE"/> </user>
