Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain]

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain]

New Contributor

Hi,

 

We have recently updated the cluster to cdh 5.5.2.

 

After the upgrading the workflows which contain a spark action are returning this error. 

 

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, requirement failed: Cannot read Hadoop config dir /var/run/cloudera-scm-agent/process/10902-yarn-NODEMANAGER.
java.lang.IllegalArgumentException: requirement failed: Cannot read Hadoop config dir /var/run/cloudera-scm-agent/process/10902-yarn-NODEMANAGER.
	at scala.Predef$.require(Predef.scala:233)

This is one of our workflows:

<workflow-app
        name="cust-info-cache-builder"
        xmlns="uri:oozie:workflow:0.5"
        xmlns:sla="uri:oozie:sla:0.2">

    <start to="build-cust-info"/>
    <action name="build-cust-info">
        <spark xmlns="uri:custom:spark-action:0.1">
            <job-tracker>${resourceManager}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>oozie.launcher.mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
                <property>
                    <name>spark.yarn.queue</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <master>yarn</master>
            <mode>cluster</mode>
            <name>Customer Information Cache Builder</name>
            <class>CustomerInformationBuilder</class>
            <jar>workflows.jar</jar>
            <spark-opts>--driver-memory 1G --executor-memory 1G --executor-cores 1 --num-executors 1 ${sparkGeneralOpts}</spark-opts>
            <arg>${1dir}</arg>
            <arg>${2dir}</arg>
            <arg>${3Dir}</arg>
        </spark>
        <ok to="end"/>
        <error to="email-error"/>
        <sla:info>
            <sla:nominal-time>${nominal_time}</sla:nominal-time>
            <sla:should-start>${10 * MINUTES}</sla:should-start>
            <sla:should-end>${30 * MINUTES}</sla:should-end>
            <sla:notification-msg>Notification Cache Building</sla:notification-msg>
            <sla:upstream-apps>UpStream Cache Building</sla:upstream-apps>
        </sla:info>
    </action>
    <action name="email-error">
        <email xmlns="uri:oozie:email-action:0.1">
            ...
        </email>
        <ok to="kill"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>  

 

We have checked yarn and oozie logs and we are not finding any clue. Does someone have a similar error?

 

Thanks in advance.

Regards,

 

2 REPLIES 2

Re: Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain]

Master Guru
I've usually observed this over Shell actions running Spark commands but not on a Spark action. Could you reconfirm if it does happen on your quoted workflow that uses a direct Spark Action?

For the shell action, while https://issues.apache.org/jira/browse/OOZIE-2481 would help in future, the workaround is to simply add the below line into the shell script:

export YARN_CONF_DIR=$HADOOP_CONF_DIR

Re: Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain]

New Contributor

Hi Harsh,

 

Thanks a lot for your response.

 

I am using a spark action: <spark xmlns="uri:custom:spark-action:0.1">

 

Do you know how to export this variable from oozie?

 

I think the problem is with YARN_CONF_DIR variable which is not being update with the proper path.

 

This YARN_CONF_DIR should have the same value that HADOOP_CONF_DIR: export YARN_CONF_DIR=$HADOOP_CONF_DIR

 

My problem is that I do not know how to do it in oozie. Do you know how to do it?

 

Thanks in advance.

Regards,