Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎02-26-2019

spark-submit Client cannot authenticate with kerberos

[ Edited ]

Hi,
I'm trying to investigate how to submit spark jobs in a kerberized cluster,
I understood that I need to user keytab file to authenticate when executing the job.
I submitted the job like that:
spark-submit --master yarn --conf spark.yarn.keytab=/tmp/cdhuser.keytab --conf spark.yarn.principal=cdhuser --class com.bezeqint.DNS_Loader.AppTest hdfs://masternode1:8020/path/to/jar_file.jar

and I get the following error:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]

and that although I did authenticate with kerberos..

Did I miss something?

Thanks,
Maor

Posts: 49
Topics: 0
Kudos: 13
Solutions: 4
Registered: ‎07-30-2018

Re: spark-submit Client cannot authenticate with kerberos

Hi,

Can you try to execute a sample spark application. Let us know the results

spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client --keytab <location>/<filename>.keytab --principal <principle name> /opt/cloudera/parcels/CDH/jars/spark-examples*.jar 1 1

Thanks
Jerry