Now that we have the baseline configuration in place for KnoxSSO, as documented in part I of the series, we are ready to configure single sign-on for Ranger. This is a nice convenience for administrators that live in both Ambari and Ranger as part of their daily platform activities. It is also a security improvement relative to local passwords. To configure Ranger for KnoxSSO, we'll need the public key for Knox. Recall that there are a few ways to obtain this, we'll use the following. openssl s_client -connect ${knoxserver}:8443 < /dev/null | openssl x509 -out /tmp/knox.crt You'll want to copy the base64 data between BEGIN CERTIFICATE and END CERTIFICATE lines. We're now ready to configure Ranger: All we need to configure is the Knox SSO provider URL and the SSO public key. The SSO public key is the copied data we just discussed from the Knox certificate. The SSO provider URL is the URL we configured in part I that corresponds to the Knox SSO topology. Now let's try to log in to Ranger using the Quick Link from Ambari. You should be seamlessly logged in as the user that authenticated to the IdP! ... View more

The HDF 3.1 release supports single-sign on to NiFi using KnoxSSO. This article will assume you've already completed setting up KnoxSSO, as discussed in part I and part II of this series. We'll further assume that NiFi has been configured for baseline security, as documented in this HCC article. Once the websso topology has been defined in your Knox configuration, the steps to make NiFi a participating application in KnoxSSO are straightforward. A couple of notes: 1) make sure nifi.security.user.login.identity.provider is blank, since you'll be using the KnoxSSO topology's authentication method--i.e., a JWT-based Federation Provider--to gain access to the NiFi UI, and 2) make sure the value for knoxsso.token.ttl is reasonable, the default is 30000 ms, or 30 s. A larger value like 36000000--or 10 hours--likely makes sense for production environments. Once the websso topology has been defined for KnoxSSO, as discussed in part I and part II of this series, we'll need to grab the Knox server's public key in order to configure NiFi as a participating application. You can use this snippet, where ${knoxserver} is the host running the Knox Gateway openssl s_client -connect ${knoxserver}:8443</dev/null| openssl x509 -out /tmp/knox.pem You can then copy the knox.pem file that gets created in /tmp to the NiFi host(s) that require this public key to verify the validity of the token signed by Knox. We'll copy knox.pem to /usr/hdf/current/nifi/conf on the NiFi host(s), for this example. We are now ready to configure NiFi, there are only three properties that are required: Please note that we should replace the nifi.security.user.knox.url value with the KnoxSSO URL specific to our environment. ... View more

Part II can be found here There are many web UIs--think Ambari, Ranger, etc.--in the HDP platform that require user authentication and traditionally every UI required a separate login. With the federation capabilities provided by Apache Knox, the HDP platform can support federated identity and a single sign-on experience for users. In particular, the flexibility of the Knox authentication and federation providers allows KnoxSSO to support generalized authentication events, via exchange of a common JWT-based token. Without the token exchange capabilities offered by KnoxSSO, each UI would need to integrate with each desired IAM solution on its own. KnoxSSO comes with its own Form-based IdP. This allows for easily integrating a form-based login with the enterprise AD/LDAP server. We will cover that in a future article, as we'll focus at present on the integration with Okta using SAML 2.0. You can find more information regarding SAML on Okta's website: https://developer.okta.com/standards/SAML/ This architecture is an example of federated identity. When a user navigates to the Ambari UI, they will be redirected to Okta to authenticate. After authenticating with their Okta credentials (and possibly an MFA mechanism), or if they've already authenticated to Okta and their session is still valid (usually within the day, as session expiration is usually 12 - 18 hours), they will be redirected back to Ambari and transparently logged in to the application. You can sign up for an Okta dev instance at https://developer.okta.com/ You will create a new application in Okta which will contain the required endpoints and metadata for verifying login requests and redirecting users back to the KnoxSSO endpoint after successful authentication. You will need the Single Sign-On URL, which will be of the form http://<KNOX_HOST>:<KNOX_PORT>/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client It's very important that you specify the URL parameter of pac4jCallback=true, otherwise the browser would get stuck in an infinite redirect loop. In Okta, you can also use this URL for Recipient URL, Destination URL, and Audience Restriction. Another thing to note on the Okta side, is that users will log in with an email address and Okta, by default, will pass the full address to KnoxSSO. We will map this to a username in the Knox configuration (alternatively, we could also have configured Okta to just send the Email Prefix). After creating the application in Okta, we are ready to configure the HDP side in Ambari. NOTE: LDAP Authentication for Ambari must have already been enabled for KnoxSSO. Please see the Ambari docs to complete this within your environment. We will pick up with Knox configuration in part two. ... View more

In previous releases of HDP, client-side caching of keys could result in unexpected behavior with WebHDFS. Consider the following steps: 1. Create two keys in ranger KMS: user1_key and user2_key 2. Add two resource based policy one per above user. User1_encr_policy: Allow the Decrypt_EEK permissions to user1 only User2_encr_policy: Allow the Decrypt_EEK permissions to user2 only. 3. Add two encryption zones. user1_zone (using user1_key) and user2_zone (using user2_key) 4. Run the following command, you may be able to access the content of test.csv file from user1_zone using user2 curl -i -L "http://sandbox.hortonworks.com:50070/webhdfs/v1/customer/user1_zone/test.csv?user.name=user2&op=OPEN" HDP-2.6.1.2 includes HADOOP-13749, which fixes the caching issue. The FS cache and KMS provider cache can be disabled by changing the configuration as follows: "fs.hdfs.impl.disable.cache", "true" dfs.client.key.provider.cache.expiry, 0 ... View more