My issue is that I am trying to run a shell action in oozie which has a simple spark-submit,  I am creeating a SparkSession with enableHiveSupport.

The error that I get when spark-submit is executed is -

• javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials

I am able to run my standalone spark-submit from the command line from any of the hosts in the cluster.

Issue happens when I try to run the same from oozie.

Reading through oozie, I have setup the hcat and hive cred and was using the cred tag for the shell action.

In the first job which is launched by oozie which is the MR job to launch the shell - I see the fiollowing - 

• HADOOP_TOKEN_FILE_LOCATION=/data/yarn/nm/usercache/dev_nifi/appcache/application_1523596809577_0129/container_1523596809577_0129_01_000002/container_tokens:

I also have the --conf spark.yarn.security.tokens.hive.enabled=false as an arg for the spark-submit to not try to get new tokens but to use the oozie token 

The error that I get when spark-submit is executed is -

• javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

I see that I am getting all the delegation tokens that are needed for the spark-submit. I even see the tokens info in the log file.

I have been struggling with this issue with little help from google searches.

My question is how to ascertain that the delegation tokens acquired by oozie is actually passed to the yarn.

I have also tried to explicitly use --keytab and --principal as an arg for spark-submit but that also did not work.

Is oozie -->shell-->spark-submit (with hive support) supported in a kerberized cluster.