Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Is Zeppelin in HDP 2.5 support multi-tenancy on a Kerberized Cluster

avatar

We are looking at setting up Zeppelin on top of Livy Server. Does the following settings pass also the kerberos authentication information.

<property>
    <name>hadoop.proxyuser.livy.groups</name>
    <value>*</value>
</property>

<property>
    <name>hadoop.proxyuser.livy.hosts</name>
    <value>*</value>
</property>

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_zeppelin-component-guide/content/install...

1 ACCEPTED SOLUTION

avatar

Hi Deepak, see my how-to tutorial:

https://community.hortonworks.com/content/kbentry/65449/ow-to-setup-a-multi-user-active-directory-ba...

If you are using self-signed certificate, Download the SSL certificate to where zeppelin is running

  1. <code>mkdir -p /etc/security/certificates

store the certificate in this directory

Import certificate for zeppelin to work with the self signed certificate.

  1. <code>cd /etc/security/certificates
  2. keytool -import -alias sampledcfieldcloud -file ad01.your.domain.name.cer -keystore /usr/jdk64/jdk1.8.0_77/jre/lib/security/cacerts
  3. keytool -list -v -keystore /usr/jdk64/jdk1.8.0_77/jre/lib/security/cacerts | grep sampledcfieldcloud

Create home directory in hdfs for the user that you will login:

  1. <code>hdfs dfs -mkdir /user/hadoopadmin
  2. hdfs dfs -chown hadoopadmin:hdfs /user/hadoopadmin

Enable multi-user zeppelin use ambari -> zeppelin notebook configs

expand the Advanced zeppelin-env and look for shiro.ini entry. Below is configuration that works with our sampledcfield Cloud.

  1. <code>[users]
  2. # List of users with their password allowed to access Zeppelin.
  3. # To use a different strategy (LDAP / Database / ...) check the shiro doc at http://shiro.apache.org/configuration.html#Configuration-INISections
  4. #admin = password1
  5. #user1 = password2, role1, role2
  6. #user2 = password3, role3
  7. #user3 = password4, role2
  8. # Sample LDAP configuration, for user Authentication, currently tested for single Realm
  9. [main]
  10. activeDirectoryRealm = org.apache.zeppelin.server.ActiveDirectoryGroupRealm
  11. #activeDirectoryRealm.systemUsername = CN=binduser,OU=ServiceUsers,DC=sampledcfield,DC=hortonworks,DC=com
  12. activeDirectoryRealm.systemUsername = binduser
  13. activeDirectoryRealm.systemPassword = xxxxxx
  14. activeDirectoryRealm.principalSuffix = @your.domain.name
  15. #activeDirectoryRealm.hadoopSecurityCredentialPath = jceks://user/zeppelin/zeppelin.jceks
  16. activeDirectoryRealm.searchBase = DC=sampledcfield,DC=hortonworks,DC=com
  17. activeDirectoryRealm.url = ldaps://ad01.your.domain.name:636
  18. activeDirectoryRealm.groupRolesMap = "CN=hadoop-admins,OU=CorpUsers,DC=sampledcfield,DC=hortonworks,DC=com":"admin"
  19. activeDirectoryRealm.authorizationCachingEnabled = true
  20. sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
  21. cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
  22. securityManager.cacheManager = $cacheManager
  23. securityManager.sessionManager = $sessionManager
  24. securityManager.sessionManager.globalSessionTimeout = 86400000
  25. #ldapRealm = org.apache.shiro.realm.ldap.JndiLdapRealm
  26. #ldapRealm.userDnTemplate = uid={0},cn=users,cn=accounts,dc=example,dc=com
  27. #ldapRealm.contextFactory.url = ldap://ldaphost:389
  28. #ldapRealm.contextFactory.authenticationMechanism = SIMPLE
  29. #sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
  30. #securityManager.sessionManager = $sessionManager
  31. # 86,400,000 milliseconds = 24 hour
  32. #securityManager.sessionManager.globalSessionTimeout = 86400000
  33. shiro.loginUrl = /api/login
  34. [roles]
  35. admin = *
  36. [urls]
  37. # anon means the access is anonymous.
  38. # authcBasic means Basic Auth Security
  39. # To enfore security, comment the line below and uncomment the next one
  40. /api/version = anon
  41. /api/interpreter/** = authc, roles[admin]
  42. /api/credential/** = authc, roles[admin]
  43. /api/configurations/** = authc, roles[admin]
  44. #/** = anon
  45. /** = authc
  46. #/** = authcBasic

Grant Livy ability to impersonate

Use Ambari to update core-site.xml, restart YARN & HDFS after making this change.

  1. <code><property>
  2. <name>hadoop.proxyuser.livy.groups</name>
  3. <value>*</value>
  4. </property>
  5. <property>
  6. <name>hadoop.proxyuser.livy.hosts</name>
  7. <value>*</value>
  8. </property>

Restart hdfs and yarn after this update.

After running the livy notebook make sure the yarn logs show the logged in user as the user that is running, hadoopadmin is the user that is logged in the zeppelin notebook. You should see 2 applications running the livy-session-X and the zeppelin app running in yarn

  1. <code>application_1478287338271_0003 hadoopadmin livy-session-0
  2. application_1478287338271_0002 zeppelin Zeppelin

View solution in original post

6 REPLIES 6

avatar

Hi Deepak, see my how-to tutorial:

https://community.hortonworks.com/content/kbentry/65449/ow-to-setup-a-multi-user-active-directory-ba...

If you are using self-signed certificate, Download the SSL certificate to where zeppelin is running

  1. <code>mkdir -p /etc/security/certificates

store the certificate in this directory

Import certificate for zeppelin to work with the self signed certificate.

  1. <code>cd /etc/security/certificates
  2. keytool -import -alias sampledcfieldcloud -file ad01.your.domain.name.cer -keystore /usr/jdk64/jdk1.8.0_77/jre/lib/security/cacerts
  3. keytool -list -v -keystore /usr/jdk64/jdk1.8.0_77/jre/lib/security/cacerts | grep sampledcfieldcloud

Create home directory in hdfs for the user that you will login:

  1. <code>hdfs dfs -mkdir /user/hadoopadmin
  2. hdfs dfs -chown hadoopadmin:hdfs /user/hadoopadmin

Enable multi-user zeppelin use ambari -> zeppelin notebook configs

expand the Advanced zeppelin-env and look for shiro.ini entry. Below is configuration that works with our sampledcfield Cloud.

  1. <code>[users]
  2. # List of users with their password allowed to access Zeppelin.
  3. # To use a different strategy (LDAP / Database / ...) check the shiro doc at http://shiro.apache.org/configuration.html#Configuration-INISections
  4. #admin = password1
  5. #user1 = password2, role1, role2
  6. #user2 = password3, role3
  7. #user3 = password4, role2
  8. # Sample LDAP configuration, for user Authentication, currently tested for single Realm
  9. [main]
  10. activeDirectoryRealm = org.apache.zeppelin.server.ActiveDirectoryGroupRealm
  11. #activeDirectoryRealm.systemUsername = CN=binduser,OU=ServiceUsers,DC=sampledcfield,DC=hortonworks,DC=com
  12. activeDirectoryRealm.systemUsername = binduser
  13. activeDirectoryRealm.systemPassword = xxxxxx
  14. activeDirectoryRealm.principalSuffix = @your.domain.name
  15. #activeDirectoryRealm.hadoopSecurityCredentialPath = jceks://user/zeppelin/zeppelin.jceks
  16. activeDirectoryRealm.searchBase = DC=sampledcfield,DC=hortonworks,DC=com
  17. activeDirectoryRealm.url = ldaps://ad01.your.domain.name:636
  18. activeDirectoryRealm.groupRolesMap = "CN=hadoop-admins,OU=CorpUsers,DC=sampledcfield,DC=hortonworks,DC=com":"admin"
  19. activeDirectoryRealm.authorizationCachingEnabled = true
  20. sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
  21. cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
  22. securityManager.cacheManager = $cacheManager
  23. securityManager.sessionManager = $sessionManager
  24. securityManager.sessionManager.globalSessionTimeout = 86400000
  25. #ldapRealm = org.apache.shiro.realm.ldap.JndiLdapRealm
  26. #ldapRealm.userDnTemplate = uid={0},cn=users,cn=accounts,dc=example,dc=com
  27. #ldapRealm.contextFactory.url = ldap://ldaphost:389
  28. #ldapRealm.contextFactory.authenticationMechanism = SIMPLE
  29. #sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
  30. #securityManager.sessionManager = $sessionManager
  31. # 86,400,000 milliseconds = 24 hour
  32. #securityManager.sessionManager.globalSessionTimeout = 86400000
  33. shiro.loginUrl = /api/login
  34. [roles]
  35. admin = *
  36. [urls]
  37. # anon means the access is anonymous.
  38. # authcBasic means Basic Auth Security
  39. # To enfore security, comment the line below and uncomment the next one
  40. /api/version = anon
  41. /api/interpreter/** = authc, roles[admin]
  42. /api/credential/** = authc, roles[admin]
  43. /api/configurations/** = authc, roles[admin]
  44. #/** = anon
  45. /** = authc
  46. #/** = authcBasic

Grant Livy ability to impersonate

Use Ambari to update core-site.xml, restart YARN & HDFS after making this change.

  1. <code><property>
  2. <name>hadoop.proxyuser.livy.groups</name>
  3. <value>*</value>
  4. </property>
  5. <property>
  6. <name>hadoop.proxyuser.livy.hosts</name>
  7. <value>*</value>
  8. </property>

Restart hdfs and yarn after this update.

After running the livy notebook make sure the yarn logs show the logged in user as the user that is running, hadoopadmin is the user that is logged in the zeppelin notebook. You should see 2 applications running the livy-session-X and the zeppelin app running in yarn

  1. <code>application_1478287338271_0003 hadoopadmin livy-session-0
  2. application_1478287338271_0002 zeppelin Zeppelin

avatar

Thanks @azeltov. To confirm. Does it also work with a Kerberized Cluster ?I am just wondering how the kerberos information is passed.

avatar

It does work in Kerberized cluster you will need to create keytabs for zeppelin and livy service account.

avatar

Thanks @azeltov . Even if we create a kerberos token for zeppelin, how does the kerberos tokens for individual users is passed ? All the access to HDFS, Spark and Hive is managed in Ranger for AD user or group and not for Zeppelin user.

avatar

You must use Livy integration for the user tokens to be passed.

avatar
Contributor

The tokens aren't passed. zeppelin authenticates itself with livy and as it is a superuser (livy.superusers) livy takes the proxyUser sent by zeppelin and becomes that user.