Could not run Oozie workflow with distcp-action

Hefei — Mon, 19 Aug 2019 12:22:55 GMT

Hi,

I am trying to run Oozie workflow job with distcp-action but getting below error:

2015-12-30 10:19:36,004  INFO CallbackServlet:520 - SERVER[centos10-82.letv.cn] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] callback for action [0000013-151228182829447-oozie-oozi-W@distcp-node]
2015-12-30 10:19:36,138  INFO DistcpActionExecutor:520 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] action completed, external ID [job_1450278216920_52402]
2015-12-30 10:19:36,152  WARN DistcpActionExecutor:523 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.DistcpMain], exception invoking main(), java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
2015-12-30 10:19:36,152  WARN DistcpActionExecutor:523 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] Launcher exception: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:234)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 9 more
2015-12-30 10:19:36,179  INFO ActionEndXCommand:520 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] ERROR is considered as FAILED for SLA
2015-12-30 10:19:36,196  INFO ActionStartXCommand:520 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@fail] Start action [0000013-151228182829447-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

and the workflow.xml file content as below:

<workflow-app xmlns="uri:oozie:workflow:0.4" name="distcp-WF">
    <start to="distcp-node"/>
    <action name="distcp-node">
        <distcp xmlns="uri:oozie:distcp-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <prepare>
<delete path="${to}"/>
</prepare>            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
               <property>
                   <name>oozie.use.system.libpath</name>
                   <value>true</value>
               </property>            </configuration>
            <arg>${from}</arg>
            <arg>${to}</arg>
        </distcp>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>DistCP failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

then, i checked the SharedLib in these path:

ls /usr/hdp/2.3.2.0-2950/oozie/share/lib/distcp

hdfs oozie ShareLib:

please help me, Thanks!

Re: Could not run Oozie workflow with distcp-action

bsaini — Wed, 30 Dec 2015 11:32:54 GMT

@Hefei Li

It appears that you have the jars at the right place but it could be a permissions issue. I see in your screen snapshots that the jar file have the 644 permissions but how about the directory containing the jar files? The directories are recommended to have 755.

This issue could occur due to incorrect value of umask (recommended value 0022)

Fix: Try setting the permissions of all directories in path to 755 and try again.

Let us know how it goes.

Re: Could not run Oozie workflow with distcp-action

Hefei — Mon, 19 Aug 2019 12:22:41 GMT

i changed all jars permissions, and the result like below:

then, i run the job again, but it still not works ok, and the error log as below:

Re: Could not run Oozie workflow with distcp-action

bsaini — Wed, 30 Dec 2015 13:50:06 GMT

@Hefei Li

Can you set the directory permissions to 755? (all the directories ,including parents, in the path that contains the jar files)

Re: Could not run Oozie workflow with distcp-action

Hefei — Wed, 30 Dec 2015 14:12:46 GMT

@bsaini

yes, i am sure.

but i have a puzzle. try it like above, is need to do on each machine in the Hadoop cluster, or just do it on Oozie server machine.

Re: Could not run Oozie workflow with distcp-action

Hefei — Thu, 31 Dec 2015 18:38:50 GMT

@bsaini

Hi bsaini. I finally sloved it.

Like you said, is a permissions issue. All the jar files which installed in HDFS must be set to 755.

My HDP version is 2.3.x, and the default Oozie Share Library is hdfs://{my_host}:8020//user/oozie/share/lib/lib_20151119202305/oozie.

Thank you for your guidance.

Re: Could not run Oozie workflow with distcp-action

bsaini — Thu, 31 Dec 2015 22:12:40 GMT

@Hefei Li

Great! Can you accept the answer then so we can close this question and others having similar issue get benefited?

question Could not run Oozie workflow with distcp-action in Archives of Support Questions (Read Only)

Could not run Oozie workflow with distcp-action

Re: Could not run Oozie workflow with distcp-action

Re: Could not run Oozie workflow with distcp-action

Re: Could not run Oozie workflow with distcp-action

Re: Could not run Oozie workflow with distcp-action

Re: Could not run Oozie workflow with distcp-action

Re: Could not run Oozie workflow with distcp-action