Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark-submit Client cannot authenticate with kerberos

spark-submit Client cannot authenticate with kerberos

New Contributor

Hi,
I'm trying to investigate how to submit spark jobs in a kerberized cluster,
I understood that I need to user keytab file to authenticate when executing the job.
I submitted the job like that:
spark-submit --master yarn --conf spark.yarn.keytab=/tmp/cdhuser.keytab --conf spark.yarn.principal=cdhuser --class com.bezeqint.DNS_Loader.AppTest hdfs://masternode1:8020/path/to/jar_file.jar

and I get the following error:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]

and that although I did authenticate with kerberos..

Did I miss something?

Thanks,
Maor

1 REPLY 1

Re: spark-submit Client cannot authenticate with kerberos

Rising Star
Hi,

Can you try to execute a sample spark application. Let us know the results

spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client --keytab <location>/<filename>.keytab --principal <principle name> /opt/cloudera/parcels/CDH/jars/spark-examples*.jar 1 1

Thanks
Jerry
Don't have an account?
Coming from Hortonworks? Activate your account here