In the previous article, we showed how to deploy a cluster running HDF 2.x or 3.x. Here we will look into enabling SSL for Apache Nifi on the cluster setup previously and optionally setup identity mappings. This approach also sets up users/authorizations using Nifi's file-based authorizer (as opposed to Ranger based authorizer).
Tested with HDF 2.x, 3.0, 3.2
1. Configure Nifi for SSL
There are 2 options for configuring SSL for Apache Nifi via Ambari:
- i). Use Nifi CA to generate self-signed certificates (good for quick start/demos)
- ii). Use existing certificates (used in production envs)
Option i) - Use Nifi Certificate Authority (CA) to generate self-signed certificates:
Assuming Nifi CA is already installed (via Ambari when you installed NiFi), you can make the below config changes in Ambari under Nifi > Configs > “Advanced nifi-ambari-ssl-config” and click Save to commit the changes:
- a) Enable SSL? Check box
- b) Clients need to authenticate? Check box
- c) NiFi CA Token - Set this to long, random value (at least 16 chars) but make sure you remember what it is set to
- d) Initial Admin Identity - set this to the long form (full DN) of identity for who your nifi admin user should be e.g. CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM (note the space after the comma)
- e) Node Identities - set this to the long form (full DN) of identity for each node running Nifi (replace CN entries below with FQDNs of nodes running Nifi...also note the space after the comma) e.g.
<property name="Node Identity 1">CN=node1.fqdn, OU=CLOUD.HORTONWORKS.COM</property> <property name="Node Identity 2">CN=node2.fqdn, OU=CLOUD.HORTONWORKS.COM</property> <property name="Node Identity 3">CN=node3.fqdn, OU=CLOUD.HORTONWORKS.COM</property>
Tip: By default the node identities are commented out using <!-- and --> tags. As you are updating this field, make sure you remove these or you changes will not take affect.
- f) NiFi CA DN suffix - in case you are not using OU=NIFI then you need to change this too (note the space after the comma) e.g.
, OU=CLOUD.HORTONWORKS.COM
- g) (Optional) You may also choose to set Identity Mapping properties at this time. These are used to normalize identities for better integration with LDAP/AD (e.g. if you wish to login as nifiadmin@CLOUD.HORTONWORKS.COM instead of CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM). Let's skip this for now...step #6 (see end of this article) is provided to show how we can switch to using these later on in the process.
Summary of above changes:
Note on identity fields above:
These are not needed to be set if you plan to use Ranger authorizer. But if you plan on logging into the Nifi UI before enabling Ranger you will need to set these. When setting these, you must make sure that on all the nodes, authorizations.xml do not contain any policies. On initial install they should already have no policies, but for example, if you made a mistake setting these first time around and want to modify the values, for the new values to take effect you will need to delete authorizations.xml on all the nodes before restarting Nifi). You can find authorizations.xml under /var/lib/nifi/conf by default (this location can be configured by ‘Nifi internal config dir’).
Troubleshooting node identities:
How will you know you made a mistake while setting node identities? Usually if the node identities field was not correctly set, when you attempt to open the Nifi UI, you will see an untrusted proxy error similar to below:
You will see some a similar 'Untrusted proxy' error in /var/log/nifi/nifi-user.log:
[NiFi Web Server-172] o.a.n.w.s.NiFiAuthenticationFilter Rejecting access to web api: Untrusted proxy CN=tsys-nifi0.field.hortonworks.com, OU=NIFI
In the above case, you would need to double check that the 'Node identity' values you provided in Ambari match the one from the log file (e.g. CN=tsys-nifi0.field.hortonworks.com, OU=NIFI) and ensure the values are not commented out. Next, you would manually delete /var/lib/nifi/conf/authorizations.xml from all nodes running Nifi and then restart Nifi service via Ambari.
Notes on Nifi CA:
If you already enabled ssl and wanted to change OU (or wanted to move CA to different node)
you can force regeneration of the certificates by either checking “NiFi CA Force Regenerate” checkbox or changing the passwords
If you had previously were not using CA and had set the passwords, but now wanted to start using CA, you can clear the passwords and check the “NiFi CA Force Regenerate” checkbox
Option ii) - Use existing certificates:
First manually copy certificates to all nodes running Nifi (e.g. under /usr/hdf/current/nifi/conf), then make the below config changes in Ambari under Nifi > Configs > “Advanced nifi-ambari-ssl-config” and and click Save to commit the changes:
- a) Enable SSL? Check box
- b) Clients need to authenticate? Check box
- c) Set Keystore and Truststore path e.g. {{nifi_config_dir}}/keystore.jks
- d) Set Keystore and Truststore type e.g. JKS
- e) Set Keystore and Truststore passwords
- f) Initial Admin Identity - set this to the long form (full DN) of identity for who your nifi admin user should be e.g. CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM
- g) Node Identities - set this to the long form (full DN) of identity for each node running Nifi (replace nodeX.fdqn with FQDNs of nodes running Nifi) e.g.
<property name="Node Identity 1">CN=node1.fqdn, OU=CLOUD.HORTONWORKS.COM</property> <property name="Node Identity 2">CN=node2.fqdn, OU=CLOUD.HORTONWORKS.COM</property> <property name="Node Identity 3">CN=node3.fqdn, OU=CLOUD.HORTONWORKS.COM</property>
- h) (Optional) You may also choose to set Identity Mapping properties at this time. Step #6 (see end of this article) is provided to show how we can switch to using these later on in the process.
2. Enable SSL for Nifi
For both options, once the above changes have been made, Ambari will prompt you to restart Nifi.
After restarting, it may take a minute for Nifi UI to come up. You can track the progress by monitoring nifi-app.log. You can do this by either tailing the log via SSH or using Logsearch:
tail -f /var/log/nifi/nifi-app.log
Another option is to run Nifi service check from Ambari. It will keep checking if the UI came up until it does:
3. Generate client certificate
In order to login to SSL-enabled Nifi, you will need to generate a client certificate and import into your browser. If you used the CA, you can use tls-toolkit that comes with Nifi CA:
- First run below from Ambari node to install the toolkit:
wget http://localhost:8080/resources/common-services/NIFI/1.0.0/package/archive.zip unzip archive.zip
- Then run below to generate keystore. You will need to pass in your values for :
- -D : pass in your “Initial Admin Identity” value
- -t: pass in your “CA token” value.
- -c: pass in the hostname of the node where Nifi CA is running:
export JAVA_HOME=/usr/java/default ./files/nifi-toolkit-*/bin/tls-toolkit.sh client -c <nifi_CA_host.fqdn> -D 'CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM' -p 10443 -t hadoop -T pkcs12
- If you pass in the wrong password, you will see an error like:
Service client error: Received response code 403 with payload {"hmac":null,"pemEncodedCertificate":null,"error":"forbidden"} - Before we can import the certificate, we will need to find the password to import. To do this, run below:
cat config.json | grep keyStorePassword
- (Optional) - The password generated above will be a long randomly generated string. If you want to change this password to one of your choosing instead, first run the below to remove the keystore/truststore:
rm -f keystore.pkcs12 truststore.pkcs12
Then edit config.json by modifying the value of “keyStorePassword" to your desired password
vi config.json
Then re-run tls-toolkit.sh as below:
./files/nifi-toolkit-*/bin/tls-toolkit.sh client -F
- At this point the keystore.pkcs12 has been generated. Rename it to keystore.p12 and transfer it (e.g. via scp) over to your local laptop.
mv keystore.pkcs12 keystore.p12
.
4. Import certificate to your browser
The exact steps depend on your OS and browser.
For example if using Chrome on Mac, use “Keychain Access” app: File > Import items > Enter password from above (you will need to type it out)
For Firefox example see here
5. Check Nifi access
Now you open Nifi UI using the Quicklink in Ambari. After selecting the certificate you imported earlier, follow the below screens to get through Chrome warnings and access the Nifi UI:
a) Select the certificate you just imported
b) Choose "Always Allow"
c) Since the certificate was self-signed, Chrome will warn you that the connection is not private. Click "Show Advanced" and click the "Proceed to <hostname>" link
d) At this point, the Nifi UI should come up. On the left, it shows 3/3, meaning all three of the Nifi nodes have joined the cluster. Note that on the top right, it shows you are logged in as "CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM"
e) The /var/log/nifi/nifi-user.log log file will also confirm the user you are getting logged in as:
o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM
f) Notice also that users.xml and authorizations.xml were created. Checking their content reveals that Nifi auto-created users and access policies for the 'Initial Admin Identity' and 'Node Identities'. More details on these files can be found here
cat /var/lib/nifi/conf/users.xml cat /var/lib/nifi/conf/authorizations.xml
With this you have successfully enabled SSL for Apache Nifi on your HDF cluster
6. (Optional) Setup Identity mappings
(Optional) If desired, we can also setup the Identity mappings to try that option as well.
- First let's remove the authorization.xml on all nifi nodes to force Nifi to re-generate them. Without doing this, you will encounter an error at login saying: "Unable to perform the desired action due to insufficient permissions"
rm /var/lib/nifi/conf/authorizations.xml
- Now make the below changes in Ambari under Nifi > Configs and click Save. (Tip: Type .dn in the textbox to Filter the fields to easily find these fields)
- nifi.security.identity.mapping.pattern.dn = ^CN=(.*?), OU=(.*?)$
- nifi.security.identity.mapping.value.dn = $1@$2
- From Ambari, restart Nifi and wait for the Nifi nodes to join back the cluster
- After about a minute, refresh the Nifi UI and notice now you are logged in as nifiadmin@CLOUD.HORTONWORKS.COM instead
- Opening /var/log/nifi/nifi-user.log confirms this:
o.a.n.w.s.NiFiAuthenticationFilter Authentication success for nifiadmin@CLOUD.HORTONWORKS.COM
- Opening users.xml, authorizations.xml shows that this time Nifi auto-created users and access policies for the 'Initial Admin Identity' and 'Node Identities' in both unmapped (e.g. CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM) and mapped (e.g. nifiadmin@CLOUD.HORTONWORKS.COM) formats:
# cat /var/lib/nifi/conf/users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users>
<user identifier="60911b91-233d-33fe-abe9-b832d8fb06fc" identity="nifiadmin@CLOUD.HORTONWORKS.COM"/>
<user identifier="dbbad79e-b7d8-30a7-963a-d152f1953343" identity="abajwa-hdf-dev-rhel7-1.openstacklocal@CLOUD.HORTONWORKS.COM"/>
<user identifier="7ee92ccf-b548-3de3-b74e-27c1e6e280ab" identity="abajwa-hdf-dev-rhel7-2.openstacklocal@CLOUD.HORTONWORKS.COM"/>
<user identifier="6ce282f4-9da7-31d4-8733-138364d88261" identity="abajwa-hdf-dev-rhel7-3.openstacklocal@CLOUD.HORTONWORKS.COM"/>
<user identifier="e3c6593b-8ab7-3e50-9778-dd662635aa8f" identity="CN=nifiadmin, OU=CLOUD.HORTONWORKS.COM"/>
<user identifier="bb834fc7-7232-3c4d-821e-3e07731100e4" identity="CN=abajwa-hdf-dev-rhel7-3.openstacklocal, OU=CLOUD.HORTONWORKS.COM"/>
<user identifier="6c6c1c9c-90b9-3fc1-9c4f-83db69f9d2b6" identity="CN=abajwa-hdf-dev-rhel7-2.openstacklocal, OU=CLOUD.HORTONWORKS.COM"/>
<user identifier="c5bb13c1-79bc-3c9e-98bc-1593985f7fd1" identity="CN=abajwa-hdf-dev-rhel7-1.openstacklocal, OU=CLOUD.HORTONWORKS.COM"/>
</users>
</tenants>- With this we have completed the setup of Identity mappings with SSL enabled Nifi
What to try next?
- Configuring other users and access policies (i.e. continuing to use Nifi's file based authorizer):
- Setup Ranger and configure Nifi Ranger plugin (i.e. switching to using Nifi's Ranger authorizer):
