HDP 2.6+ - Configuring Zeppelin for Active Directory user authentication & authorization

[#1]

Update:

This is an update to my previous article on the same topic. This covers the new features added in HDP 2.6 (and Zeppelin 0.7).

Motivation:

Starting HDP 2.6, a new Shiro configuration implementation has been added in Zeppelin to handle LDAP/Active Directory authentication and authorization. It fixes lot of known issues (Bind issue, limited search/filter options, Group based authorization etc.) present in earlier versions and this should be used for any kind of LDAP/AD authentication + authorization going forward.

Configuration:

1. While most of the configuration steps remain same from the previous article, the following "shiro_init_content" is where the most of the magic happen:

Note: Before pasting this configuration in your Zeppelin configuration, please change the Active Directory details to suit your AD environment.

# Sample LDAP configuration, for Active Directory user Authentication, currently tested for single Realm
[main]
ldapRealm=org.apache.zeppelin.realm.LdapRealm
ldapRealm.contextFactory.systemUsername=cn=ldap-reader,ou=ServiceUsers,dc=lab,dc=hortonworks,dc=net
ldapRealm.contextFactory.systemPassword=SomePassw0rd
ldapRealm.contextFactory.authenticationMechanism=simple
ldapRealm.contextFactory.url=ldap://ad.somedomain.net:389
# Ability to set ldap paging Size if needed; default is 100
ldapRealm.pagingSize=200
ldapRealm.authorizationEnabled=true
ldapRealm.searchBase=OU=CorpUsers,DC=lab,DC=hortonworks,DC=net
ldapRealm.userSearchBase=OU=CorpUsers,DC=lab,DC=hortonworks,DC=net
ldapRealm.groupSearchBase=OU=CorpUsers,DC=lab,DC=hortonworks,DC=net
ldapRealm.userObjectClass=person
ldapRealm.groupObjectClass=group
ldapRealm.userSearchAttributeName = sAMAccountName
# Set search scopes for user and group. Values: subtree (default), onelevel, object
ldapRealm.userSearchScope = subtree
ldapRealm.groupSearchScope = subtree
ldapRealm.userSearchFilter=(&(objectclass=person)(sAMAccountName={0}))
ldapRealm.memberAttribute=member
# Format to parse & search group member values in 'memberAttribute'
ldapRealm.memberAttributeValueTemplate=CN={0},OU=CorpUsers,DC=lab,DC=hortonworks,DC=net
# No need to give userDnTemplate if memberAttributeValueTemplate is provided
#ldapRealm.userDnTemplate=
# Map from physical AD groups to logical application roles
ldapRealm.rolesByGroup = "hadoop-admins":admin_role,"hadoop-users":hadoop_users_role
# Force usernames returned from ldap to lowercase, useful for AD
ldapRealm.userLowerCase = true

# Enable support for nested groups using the LDAP_MATCHING_RULE_IN_CHAIN operator
ldapRealm.groupSearchEnableMatchingRuleInChain = true



sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
### If caching of user is required then uncomment below lines
cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
securityManager.cacheManager = $cacheManager


securityManager.sessionManager = $sessionManager
securityManager.realms = $ldapRealm
# 86,400,000 milliseconds = 24 hour
securityManager.sessionManager.globalSessionTimeout = 86400000
shiro.loginUrl = /api/login


[urls]
# This section is used for url-based security.
# You can secure interpreter, configuration and credential information by urls. Comment or uncomment the below urls that you want to hide.
# anon means the access is anonymous.
# authc means Form based Auth Security
# To enfore security, comment the line below and uncomment the next one
#/api/version = anon
/api/interpreter/** = authc, roles[admin_role,hadoop_users_role]
/api/configurations/** = authc, roles[admin_role]
/api/credential/** = authc, roles[admin_role,hadoop_users_role]
#/** = anon
/** = authc

Lets discuss the new configuration options here:

2. ldapRealm.rolesByGroup = "hadoop-admins":admin_role,"hadoop-users":hadoop_users_role

This line maps the AD groups "hadoop-admins" and "hadoop-users" to custom roles which can be used in [urls] section to control access to various Zeppelin users. Note that the short group names are to be used instead of fully qualified names like "cn=hadoop-admins,OU=CorpUsers,DC=lab,DC=hortonworks,DC=net". The role names can be set to any name but the same names should be used in the [urls] section.

3. ldapRealm.groupSearchEnableMatchingRuleInChain = true

A very powerful option to search all the groups that a given user is member of in a single query. An LDAP search query with this option traverses the LDAP group hierarchy till the root to find out all the groups. Specially useful for nested groups. More info can be found here. Caution : This option can cause performance overhead (slow to log in etc.) if LDAP hierarchy is not setup optimally.

4. ldapRealm.userSearchFilter=(&(objectclass=person)(sAMAccountName={0}))

Use this search filter to limit scope of user results when looking for user's Distinguished Name (DN). This is used only If userSearchBase and userSearchAttributeName are defined. If these two are not defined, then userDnTemplate is used to look for user's DN.

[#2]

You are missing a closing parentheses on property ldapRealm.userSearchFilter

[#3]

First of all, Nic Swart's comment is VERY IMPORTANT! Thanks much. Without the extra parentheses it just doesn't work (doesn't bother to tell you that the syntax of the ldap query is wrong, grrr!)

Second, To be clear, the ldapRealm.xxx settings are recommended for use with Active Directory starting with Zeppelin 0.7. This can be a little confusing.