Oozie Spark action with a specific principal in a kerberzied cluster

[#1]

Need to run oozie workflow with spark-action. The spark-action should be running with a different user from the user who has done the kinit while submitting the oozie job.

Can able to do this using spark-submit by passing --principal and --keytab options.

Tried to pass the same thing on the oozie spark action by adding them in <spark-opts>

But it's failing with the following exception

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, Login failure for tempUser1@REALM from keytab /etc/security/keytabs/tempUser1.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user

java.io.IOException: Login failure for tempUser1@REALM from keytab /etc/security/keytabs/tempUser1.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user

Here's my workflow:

<workflow-app xmlns="uri:oozie:workflow:0.3" name="spark-wf">
    <start to="spark-node"/>
    <action name="spark-node">
        <spark xmlns="uri:oozie:spark-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <master>yarn-cluster</master>
            <name>${jobName}</name>
            <class>${className}</class>
            <jar>${workflowAppUri}/${jarPath}</jar>
            <spark-opts>--executor-memory ${executorMemory} --executor-cores ${executorCores} --num-executors ${numExecutors} --driver-java-options ${driverJavaOptions} --principal ${kerbPrincipal} --keytab ${kerbKeytab}</spark-opts>
            <arg>${arg1}</arg>
            <arg>${arg2}</arg>
        </spark>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

Is there any way that I can execute the oozie spark action on a different user than the user who has done the kinit on the machine?

[#2]

Hi,

For this I would request you to place the keytab file in HDFS and just reference the name in the <spark-opts>

Example

<spark-opts> --principal <abc> --keytab <abc.keytab> </spark-opts>
<file> <path of HDFS keytab></file>

NOTE:- Do add the <file> tag which will be pointing to the location of keytab on HDFS.

This will localize the keytab file and will use in the oozie spark action.

Kindly try the above and let us know how it goes.

Regards

Nitish

[#3]

You can try to configure it like this inside Oozie Spark action:

...
<configuration>
    <property>
        <name>spark.yarn.keytab</name>
        <value>path_to_keytab</value>
    </property>
    <property>
        <name>spark.yarn.principal</name>
        <value>principal@REALM.COM </value>
    </property>
</configuration>
...