In the previous articles, we showed how to deploy an HDF 2.x/3.0 cluster, enable SSL for Nifi and setup the Ranger Nifi plugin. Here we will build on the same cluster and show how to enable kerberos using Active Directory.
Summary
To achieve this, the high level steps we will follow are:
- Setup certificate trust for HDF nodes
- Run Ambari security wizard
- Create Ranger policy for nifiadmin user
- Delete certificate
- Login to Nifi using AD principal credentials
Pre-requisites
You have correctly setup AD as described here
- Active Directory setup with domain: CLOUD.HORTONWORKS.COM
- AD already preconfigured with LDAPS
- Certificate (.crt) used to enable LDAPS is available
- OU created where HDF principals will be created
- hadoop user has permission to write principals to above OU
- nifiadmin user created in AD (optionally synced over to Ranger)
- Test to ensure you can access AD over LDAPS using hadoopadmin user succeeds:
ldapsearch -H ldaps://sme-security-ad03.cloud.hortonworks.com:636 -D hadoopadmin@cloud.hortonworks.com -w BadPass#1
Steps
1. Setup trust for all HDF nodes using the AD certificate
#run on all HDF nodes before running security wizard using AD
ad_ip=xx.xx.xx.xx ##replace with IP of your AD
cert_url=http://someurl/mycertificate.crt ## replace with location of exported AD certificate
echo "${ad_ip} ad01.lab.hortonworks.net ad01" | sudo tee -a /etc/hosts
sudo yum -y install openldap-clients ca-certificates
#instead of downloading the cert, you could also manually transfer the .cert file to below location
sudo curl -sSL "${cert_url}" -o /etc/pki/ca-trust/source/anchors/hortonworks-net.crt
sudo update-ca-trust force-enable
sudo update-ca-trust extract
sudo update-ca-trust check
# edit /etc/openldap/ldap.conf to include LDAP url and base
sudo tee -a /etc/openldap/ldap.conf > /dev/null << EOF
TLS_CACERT /etc/pki/tls/cert.pem
URI ldaps://ad01.lab.hortonworks.net ldap://ad01.lab.hortonworks.net
BASE dc=cloud,dc=hortonworks,dc=com
EOF
#test using openssl - should return 0
openssl s_client -connect ad01:636 </dev/null
#test using ldapsearch
ldapsearch -H ldaps://sme-security-ad03.cloud.hortonworks.com:636 -D nifiadmin@cloud.hortonworks.com -w BadPass#12. Run Ambari Security Wizard
Launch security wizard via Ambari (under Admin > Kerberos) and enter below:
The ‘Configure Kerberos’ page is the only one you will need to update. Enter the below then click Next on all remaining screens.
- KDC host: FQDN of AD
- Realm name: CLOUD.HORTONWORKS.COM
- Kadmin host: FQDN of AD node
- Admin principal: hadoopadmin@cloud.hortonworks.com
- Password: BadPass#1
On ‘Configure Identities’ page, users will be shown the option to customize the keytabs/principals for all components:
The Nifi ones are under Advanced tab:
Click Next to proceed using the default keytab/principal names
Click Next to proceed through all remaining steps of the wizard.
What’s happening to Nifi under the covers when security wizard runs?
a) NiFi principal and keytabs will be automatically be created/distributed across the cluster where needed by Ambari
b) Kerberos-related nifi.properties fields will automatically be updated:
- NiFi.kerberos.service.principal
- NiFi.kerberos.keytab.location
- NiFi.kerberos.krb5.file
- NiFi.kerberos.authentication.expiration
c) Login provider will also be switched to kerberos under the covers
d) As part of the process, other HDF components were also kerberized including ‘Ambari Infra’ service. This mean that Ranger audits are now being written to kerberized Solr
After security wizard completes, NiFi’s kerberos details will appear alongside other components (under Admin > Kerberos). At this point, Kerberos security will be enabled for all components running on the cluster:
On a node running Nifi, you can verify the keytab was generated and list its principal
# klist -kt /etc/security/keytabs/nifi.service.keytab Keytab name: FILE:/etc/security/keytabs/nifi.service.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@CLOUD.HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@CLOUD.HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@CLOUD.HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@CLOUD.HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@CLOUD.HORTONWORKS.COM
You can also verify the nifi configs for kerberos were automatically populated:
# cat /etc/nifi/conf/nifi.properties | grep kerberos nifi.kerberos.krb5.file=/etc/krb5.conf nifi.kerberos.service.keytab.location=/etc/security/keytabs/nifi.service.keytab nifi.kerberos.service.principal=nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@CLOUD.HORTONWORKS.COM nifi.kerberos.spnego.authentication.expiration=12 hours nifi.kerberos.spnego.keytab.location=/etc/security/keytabs/spnego.service.keytab nifi.kerberos.spnego.principal=HTTP/abajwa-hdf-qe-hdfsecured-1.openstacklocal@CLOUD.HORTONWORKS.COM nifi.security.user.login.identity.provider=kerberos-provider
You can also verify that the login-identity-provider or Nifi has now been switched to kerberos
# tail /etc/nifi/conf/login-identity-providers.xml <provider> <identifier>kerberos-provider</identifier> <class>org.apache.nifi.kerberos.KerberosProvider</class> <property name="Default Realm">HORTONWORKS.COM</property> <property name="Authentication Expiration">12 hours</property> </provider>
3. Login to Nifi UI without certificate
Now that kerberos is enabled, lets try to login without using certificate
- Make sure nifiadmin user exists in Ranger (if you ran Ranger sync earlier this should have been imported already).
- If not, create the user in Ranger by navigating to below url and entering below
http://<Ranger_node>:6080/index.html#!/user/create
- Create Ranger policy for new user
- In Ranger, under ‘Access Manager, click ‘HDF-nifi’
- Click Edit button on the /* policy we previously added nifiadmin@CLOUD.HORTONWORKS.COM to
- Add the newly created nifiadmin user to the policy, and click Save
- Delete previously imported .p12 certificates from your browser
- e.g. if using Chrome on OSX you can delete previously imported certificates using ‘Keychain Access’ application
- Restart Chrome and open Nifi UI. It should now display a login page
- If not, try opening “Incognito Window”
- Enter username as nifiadmin and the password you set
- The Nifi UI should open now and you will be logged in as that user
- You can see who you are logged in as by checking top-right corner of Nifi UI
This completes the tutorial. If you made it this far in the series, congratulations! You have successfully:
- Deployed HDF 2.0
- Enabled SSL for Nifi and explored file-based authorization for Nifi
- Installed Ranger and switched to Ranger-based authorization for Nifi
- Enabled kerberos for your HDF cluster using Active Directory
- Logged into Nifi using AD credentials
