Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Who agreed with this topic

Spark-Hive Application: SASL Negotiation Failure with Kerberos on a Cluster

Labels:
avatar
author-rank Rohan44
Explorer

Created ‎08-05-2023 05:36 AM

I'm having an issue with a Spark-Hive application running on a Kerberos cluster. I receive a javax.security.sasl.SaslException: GSS initiate failed error, which appears to be caused by not finding any Kerberos tgt.
 
Here's the error log:
 
    23/08/04 22:56:55 INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
    23/08/04 22:56:55 INFO HiveClientImpl: Attempting to login to Kerberos using principal: [email protected] and keytab: hdfs01.keytab-2ca1f730-bef7-4166-90ce-67317c75c793
    23/08/04 22:56:55 INFO UserGroupInformation: Login successful for user [email protected] using keytab file hdfs01.keytab-2ca1f730-bef7-4166-90ce-67317c75c793
    23/08/04 22:56:55 INFO metastore: Trying to connect to metastore with URI thrift://master3.abc.xyz.com:9083"
    23/08/04 22:56:55 ERROR TSaslTransport: SASL negotiation failure
    javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at org.apac...
 
I am submitting my Spark job as follows:
 
    spark-submit \
    --name TestKerberous \
    --num-executors 2 \
    --driver-java-options "-Djava.security.auth.login.config=./key_fin.conf" \
    --driver-java-options "-Dsun.security.krb5.debug=true" \
    --conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./key_fin.conf"\
    --files=/etc/spark/conf/hive-site.xml,/etc/hadoop/conf/yarn-site.xml,/etc/hadoop/conf/hdfs-site.xml,/etc/hadoop/conf/core-site.xml \
    --conf "spark.hadoop.hive.metastore.kerberos.principal=HTTP/[email protected]" \
    --conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./key.conf" \
    --conf -Djavax.security.auth.useSubjectCredsOnly=false \
    --conf spark.executorEnv.KRB5_CONFIG=/etc/krb5.conf \
    --conf spark.driverEnv.KRB5_CONFIG=/etc/krb5.conf \
    --conf "spark.hadoop.hive.metastore.sasl.enabled=true" \
    --conf "spark.hadoop.hive.security.authorization.enabled=true" \
    --conf "spark.hadoop.hive.metastore.execute.setugi=true" \
    --conf spark.sql.hive.convertMetastoreParquet=false \
    --conf spark.home=/usr/hdp/current/spark2-client \
    --conf spark.sql.warehouse.dir=/apps/hive/warehouse \
    --conf spark.sql.catalogImplementation=hive \
    --conf spark.yarn.keytab=/etc/security/keytabs/hdfs01.keytab \
    --conf [email protected] \
    --conf spark.serializer=org.apache.spark.serializer.KryoSerializer  \
    --master yarn --deploy-mode cluster --driver-cores 2 --driver-memory 2G --executor-cores 2 --executor-memory 2G --supervise \
    --class <CLASS_NAME> \
    <JAR_FILE>\
    "<Hive Jdbc Url>" "thrift://master3.abc.xyz.com:9083" "/apps/hive/warehouse"
 
 
I would really appreciate it if anyone could help me diagnose what might be going wrong and how to resolve this issue.
 
Thank you in advance for any insights you can provide
5,371
0 Kudos
1
Who agreed with this topic