Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

New Contributor
 
9 REPLIES 9

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

New Contributor

spark-submit works completely fine and executes the query successfully, but if the same is called from oozie workflow as a action. we get the below error. "No such file or directory".

Caused by: java.io.IOException: error=2, No such file or directory
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw exception, java.io.IOException: Cannot run program "./spark-submit" (in directory "/usr/hdp/current/spark2-client/bin"): error=2, No such file or directory
org.apache.oozie.action.hadoop.JavaMainException: java.io.IOException: Cannot run program "./spark-submit" (in directory "/usr/hdp/current/spark2-client/bin"): error=2, No such file or directory

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

Expert Contributor
@Fasil Ahamed

Does all the Oozie client nodes and NodeManagers have spark2-client installed?

Thanks

Venkat

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

New Contributor

Yes , oozie client and spark2 client installed on all the node manager nodes.

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

New Contributor

Yes , oozie client and spark2 client installed on all the node manager nodes.

any update here, as we see the issue is intermittent & sometimes the sqoop action throws the following error "No such file or directory"?

Quick help would be much appreciated.

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

Expert Contributor

can you please provide job.properties and workflow.xml files and also how the spark submit is called i.e. are we using Oozie spark action or are we calling the spark-submit from a shell action?

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

New Contributor

job.properties.

NameNode=hdfs://pap-hadoop-1:8020

jobTracker=pap-hadoop-2:8032

queueName=default examplesRoot=examplesoozie oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/core-services/workflow/Workflow.xml

Workflow.xml

<workflow-app xmlns="uri:oozie:workflow:0.1" name="pySparkTest">

<start to="spark_start"/> <action name="spark_start">

<java>

<job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node>

<configuration>

<property>

<name>mapred.queue.name</name>

<value>default</value>

</property>

<property>

<name>mapreduce.reduce.memory.mb</name>

<value>1024</value>

</property>

<property>

<name>mapreduce.map.memory.mb</name>

<value>1024</value>

</property>

<property>

<name>oozie.launcher.mapreduce.map.memory.mb</name>

<value>1024</value>

</property>

<property>

<name>oozie.launcher.yarn.app.mapreduce.am.resource.mb</name>

<value>1024</value>

</property>

</configuration>

<main-class>spark.SparkLauncher</main-class>

<arg>./spark-submit</arg>

<arg>--master</arg>

<arg>yarn</arg>

<arg>--deploy-mode</arg>

<arg>client</arg>

<arg>sparktest.py</arg>

<arg>scripts/sparktest.py</arg>

</java>

<ok to="end"/>

<error to="fail"/>

</action> <kill name="fail"> <message>Workflow Killed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill>

<end name="end"/>

</workflow-app>

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

Expert Contributor

It is not right way to execute spark action. It is failing because it is not able to find spark-submit command in cache directory while launching it.

Instead use spark action:

Link - https://gist.github.com/rajkrrsingh/71f43afaac098428dc614d50ca0293ac

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

New Contributor

Thanx @schhabra,

While i give a try with the process of shared link, i would like to highlight a note that the issue is intermittent & sometimes the same issue occurs for sqoop action "No Such file or directory" . is oozie missing on any jars or expecting any specific property?

Re: oozie launcher fails for spark-submit action, Caused by: java.io.IOException: error=2, No such file or directory

New Contributor

Hi All,

I figured out the fix , oozie is running the shell actions which is making YARN to call the necessary programs underneath. Since one of my Host did not have the spark-submit programme installed(whenever there was a call from YARN on that node, it used to fail.). I configured the programme on the missing host.

Now everything is running smooth. :)

Don't have an account?
Coming from Hortonworks? Activate your account here