Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Zeppelin notebook with spark interpreter impersonation failing

Highlighted

Zeppelin notebook with spark interpreter impersonation failing

New Contributor

Hi,


This is my script:

%spark2.pyspark

import math
print("The value of pi is", math.pi)


This is the output:

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: zeppelin is not allowed to impersonate mansop
    at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1497)
    at org.apache.hadoop.ipc.Client.call(Client.java:1443)
    at org.apache.hadoop.ipc.Client.call(Client.java:1353)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
    at com.sun.proxy.$Proxy16.getClusterMetrics(Unknown Source)
    at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:259)
    at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)


This is my Zeppelin impersonation config:

export SPARK_HOME=/usr/hdp/current/spark2-client/
export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH
export PYTHONPATH=$SPARK_HOME/python/lib/py4j-0.10.7-src.zip:$PYTHONPATH
export SPARK_YARN_USER_ENV="PYTHONPATH=${PYTHONPATH}"
export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c '
export ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=${ZEPPELIN_IMPERSONATE_USER}


This is my HDFS impersonation config:

hadoop.proxyuser.hdfs.groups = *
hadoop.proxyuser.hdfs.hosts = *
hadoop.proxyuser.hive.groups = *
hadoop.proxyuser.hive.hosts = *
hadoop.proxyuser.livy.groups = *
hadoop.proxyuser.livy.hosts = *
hadoop.proxyuser.root.groups = *
hadoop.proxyuser.root.hosts = *


What am I mising?


Thank you

Don't have an account?
Coming from Hortonworks? Activate your account here