Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oozie Spark action with a specific principal in a kerberzied cluster

Oozie Spark action with a specific principal in a kerberzied cluster

Need to run oozie workflow with spark-action. The spark-action should be running with a different user from the user who has done the kinit while submitting the oozie job.

Can able to do this using spark-submit by passing --principal and --keytab options.


Tried to pass the same thing on the oozie spark action by adding them in <spark-opts>

But it's failing with the following exception

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, Login failure for tempUser1@REALM from keytab /etc/security/keytabs/tempUser1.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user

java.io.IOException: Login failure for tempUser1@REALM from keytab /etc/security/keytabs/tempUser1.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user


Here's my workflow:

<workflow-app xmlns="uri:oozie:workflow:0.3" name="spark-wf">
    <start to="spark-node"/>
    <action name="spark-node">
        <spark xmlns="uri:oozie:spark-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <master>yarn-cluster</master>
            <name>${jobName}</name>
            <class>${className}</class>
            <jar>${workflowAppUri}/${jarPath}</jar>
            <spark-opts>--executor-memory ${executorMemory} --executor-cores ${executorCores} --num-executors ${numExecutors} --driver-java-options ${driverJavaOptions} --principal ${kerbPrincipal} --keytab ${kerbKeytab}</spark-opts>
            <arg>${arg1}</arg>
            <arg>${arg2}</arg>
        </spark>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

Is there any way that I can execute the oozie spark action on a different user than the user who has done the kinit on the machine?

1 REPLY 1
Highlighted

Re: Oozie Spark action with a specific principal in a kerberzied cluster

Expert Contributor

Hi,

 

For this I would request you to place the keytab file in HDFS and just reference the name in the <spark-opts>

 

Example

 

<spark-opts> --principal <abc> --keytab <abc.keytab> </spark-opts>
<file> <path of HDFS keytab></file>

 

NOTE:- Do add the <file> tag which will be pointing to the location of keytab on HDFS.

 

This will localize the keytab file and will use in the oozie spark action.

 

Kindly try the above and let us know how it goes.

 

Regards

Nitish

Don't have an account?
Coming from Hortonworks? Activate your account here