In the previous articles, we showed how to deploy an HDF 2.0 cluster, enable SSL for Nifi and setup the Ranger Nifi plugin. Now we will build on the same cluster and show how to enable kerberos from Ambari using MIT KDC.
Summary
To achieve this, the high level steps we will follow are:
- Setup MITC KDC
- Run Ambari security wizard
- Create principal for nifiadmin user in KDC
- Create Ranger policy for nifiadmin user
- Delete certificate from browser
- Login to Nifi using KDC principal credentials
Steps
1. Setup MIT KDC
High level steps to setup KDC:
- Install KDC rpms
- Configure KDC (krb5.conf)
- Create KDC database
- Start krb5kdc/kadmin services
- Create admin principal
- Make user an administrator by adding to kadm5.acl
- Restart krb5kdc/kadmin services
Script to automate KDC setup (run below on Ambari node)
export realm=HORTONWORKS.COM export domain=hortonworks.com export kdcpassword="BadPass#1" curl -sSL https://gist.github.com/abajwa-hw/f8b83e1c12abb1564531e00836b098fa/raw | sudo -E sh
Test KDC is up by running below on Ambari node:
kadmin -p admin/admin -w BadPass#1 -r HORTONWORKS.COM -q "get_principal admin/admin"
2. Run Ambari Security Wizard
Launch security wizard via Ambari (under Admin > Kerberos) and enter below:
The ‘Configure Kerberos’ page is the only one you will need to update. Enter the below then click Next on all remaining screens.
- KDC host: FQDN of KDC (Ambari) node
- Realm name: HORTONWORKS.COM
- Kadmin host: FQDN of KDC (Ambari) node
- Admin principal: admin/admin
- Password: BadPass#1
On ‘Configure Identities’ page, users will be shown the option to customize the keytabs/principals for all components:
The Nifi ones are under Advanced tab:
Click Next to proceed using the default keytab/principal names
Click Next to proceed through all remaining steps of the wizard.
What’s happening to Nifi under the covers when security wizard runs?
a) NiFi principal and keytabs will be automatically be created/distributed across the cluster where needed by Ambari
b) Kerberos-related nifi.properties fields will automatically be updated:
- NiFi.kerberos.service.principal
- NiFi.kerberos.keytab.location
- NiFi.kerberos.krb5.file
- NiFi.kerberos.authentication.expiration
c) Login provider will also be switched to kerberos under the covers
d) As part of the process, other HDF components were also kerberized including ‘Ambari Infra’ service. This mean that Ranger audits are now being written to kerberized Solr
After security wizard completes, NiFi’s kerberos details will appear alongside other components (under Admin > Kerberos). At this point, Kerberos security will be enabled for all components running on the cluster:
On a node running Nifi, you can run below commands to:
...verify the keytab was generated and list its principal
# klist -kt /etc/security/keytabs/nifi.service.keytab Keytab name: FILE:/etc/security/keytabs/nifi.service.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@HORTONWORKS.COM 1 09/28/2016 04:55:08 nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@HORTONWORKS.COM
...verify the nifi configs for kerberos were automatically populated:
# cat /etc/nifi/conf/nifi.properties | grep kerberos nifi.kerberos.krb5.file=/etc/krb5.conf nifi.kerberos.service.keytab.location=/etc/security/keytabs/nifi.service.keytab nifi.kerberos.service.principal=nifi/abajwa-hdf-qe-hdfsecured-1.openstacklocal@HORTONWORKS.COM nifi.kerberos.spnego.authentication.expiration=12 hours nifi.kerberos.spnego.keytab.location=/etc/security/keytabs/spnego.service.keytab nifi.kerberos.spnego.principal=HTTP/abajwa-hdf-qe-hdfsecured-1.openstacklocal@HORTONWORKS.COM nifi.security.user.login.identity.provider=kerberos-provider
...verify that the login-identity-provider or Nifi has now been switched to kerberos
# tail /etc/nifi/conf/login-identity-providers.xml <provider> <identifier>kerberos-provider</identifier> <class>org.apache.nifi.kerberos.KerberosProvider</class> <property name="Default Realm">HORTONWORKS.COM</property> <property name="Authentication Expiration">12 hours</property> </provider>
3. Login to Nifi UI without certificate
Now that kerberos is enabled, lets try to login without using certificate
- First create a principal in KDC for nifiadmin. From the node running KDC (same one as Ambari) run below and enter your desired password (e.g. BadPass#1):
kadmin.local -q "addprinc nifiadmin"
- Create the user in Ranger by navigating to below url and entering below
http://<Ranger_node>:6080/index.html#!/user/create
The username should be in the format userprinc@KDC_REALM (e.g. nifiadmin@HORTONWORKS.COM)
- Create Ranger policy for new user
- In Ranger, under ‘Access Manager, click ‘HDF-nifi’
- Click Edit button on the policy we previously added nifiadmin@HORTONWORKS to
- Add the newly created nifiadmin@HORTONWORKS.COM’ user to the policy, and click Save
- Delete previously imported .p12 certificates from your browser
- e.g. if using Chrome on OSX you can delete previously imported certificates using ‘Keychain Access’ application
- Restart Chrome and open Nifi UI. It should now display a login page
- If not, try opening “Incognito Window”
- Enter username as nifiadmin@HORTONWORKS.COM and the password you set
- The Nifi UI should open now and you will be logged in as that user
- You can see who you are logged in as by checking top-right corner of Nifi UI
This completes the tutorial. If you made it this far in the series, congratulations! You have successfully:
- Deployed HDF 2.0
- Enabled SSL for Nifi and explored file-based authorization for Nifi
- Installed Ranger and switched to Ranger-based authorization for Nifi
- Enabled kerberos for your HDF cluster
- Logged into Nifi using KDC credentials
