Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Contributor

Livy: Livy is an open source REST interface for interacting with Spark. Authorized users can launch a Spark session and submit code. Two different users can access their own private data and session, and they can collaborate on a notebook. Only the Livy server can submit a job securely to a Spark session.

Steps to follow to configure livy interpreter to work with secure HDP cluster:

  1. Setup proxy for livy interpreter in core-site.xml
Go to Ambari->HDFS->config->customer-core-site and add below properties: 

hadoop.proxyuser.livy.groups=* 

hadoop.proxyuser.livy.hosts=*

2. Configure livy interpreter in Zeppelin and add below configurations:

livy.superusers=zeppelin-spark

Note - The value for livy.superusers should be your zeppelin principal. That would be zeppelin-{$Cluster_name} For example, in this case you can find it by running below command:

klist -kt /etc/security/keytabs/zeppelin.server.kerberos.keytab

Keytab name: FILE:/etc/security/keytabs/zeppelin.server.kerberos.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   1 11/15/16 17:33:16 zeppelin-spark@HWX.COM
   1 11/15/16 17:33:16 zeppelin-spark@HWX.COM
   1 11/15/16 17:33:16 zeppelin-spark@HWX.COM
   1 11/15/16 17:33:16 zeppelin-spark@HWX.COM
   1 11/15/16 17:33:16 zeppelin-spark@HWX.COM

zeppelin-spark will be your superuser for livy interpreter.

*Make sure this will match with livy.superusers in livy-conf file.

livy.impersonation.enabled=true  //this configuration should also be present in livy-conf. 
livy.server.access_control.enabled=true 
livy.server.access_control.users=livy,zeppelin 
livy.server.auth.type=kerberos 
livy.server.auth.kerberos.keytab=/etc/security/keytabs/spnego.service.keytab 
livy.server.auth.kerberos.principal=HTTP/spark-1.hwx.com@HWX.COM 
livy.server.launch.kerberos.keytab=/etc/security/keytabs/livy.service.keytab 
livy.server.launch.kerberos.principal=livy/spark-1.hwx.com@HWX.COM

Note - To configure Zeppelin with authentication for Livy you need to set the following in the interpreter settings:

zeppelin.livy.principal=zeppelin-spark@HWX.COM 
zeppelin.livy.keytab=/etc/security/keytabs/zeppelin.service.keytab

3. Make sure zeppelin.livy.url is pointing to hostname not IP address :

zeppelin.livy.url=http://spark-3.hwx.com:8998

4. After saving configuration changes in livy interpreter, Please restart interpreter to see the affect.

5,447 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎01-26-2017 09:42 PM
Updated by:
 
Contributors
Top Kudoed Authors