Support Questions

Find answers, ask questions, and share your expertise

Could not run Oozie workflow with distcp-action

avatar
Rising Star

Hi,

I am trying to run Oozie workflow job with distcp-action but getting below error:

2015-12-30 10:19:36,004  INFO CallbackServlet:520 - SERVER[centos10-82.letv.cn] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] callback for action [0000013-151228182829447-oozie-oozi-W@distcp-node]
2015-12-30 10:19:36,138  INFO DistcpActionExecutor:520 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] action completed, external ID [job_1450278216920_52402]
2015-12-30 10:19:36,152  WARN DistcpActionExecutor:523 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.DistcpMain], exception invoking main(), java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
2015-12-30 10:19:36,152  WARN DistcpActionExecutor:523 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] Launcher exception: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:234)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.DistcpMain not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 9 more
2015-12-30 10:19:36,179  INFO ActionEndXCommand:520 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@distcp-node] ERROR is considered as FAILED for SLA
2015-12-30 10:19:36,196  INFO ActionStartXCommand:520 - SERVER[centos10-82.letv.cn] USER[oozie] GROUP[-] TOKEN[] APP[distcp-WF] JOB[0000013-151228182829447-oozie-oozi-W] ACTION[0000013-151228182829447-oozie-oozi-W@fail] Start action [0000013-151228182829447-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

and the workflow.xml file content as below:

<workflow-app xmlns="uri:oozie:workflow:0.4" name="distcp-WF">
    <start to="distcp-node"/>
    <action name="distcp-node">
        <distcp xmlns="uri:oozie:distcp-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <prepare>
<delete path="${to}"/>
</prepare>            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
               <property>
                   <name>oozie.use.system.libpath</name>
                   <value>true</value>
               </property>            </configuration>
            <arg>${from}</arg>
            <arg>${to}</arg>
        </distcp>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>DistCP failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>
        

then, i checked the SharedLib in these path:

  • ls /usr/hdp/2.3.2.0-2950/oozie/share/lib/distcp

1120-1.png

  • hdfs oozie ShareLib:
  • 1118-2.png

please help me, Thanks!

1 ACCEPTED SOLUTION

avatar

@Hefei Li

It appears that you have the jars at the right place but it could be a permissions issue. I see in your screen snapshots that the jar file have the 644 permissions but how about the directory containing the jar files? The directories are recommended to have 755.

This issue could occur due to incorrect value of umask (recommended value 0022)

Fix: Try setting the permissions of all directories in path to 755 and try again.

Let us know how it goes.

View solution in original post

6 REPLIES 6

avatar

@Hefei Li

It appears that you have the jars at the right place but it could be a permissions issue. I see in your screen snapshots that the jar file have the 644 permissions but how about the directory containing the jar files? The directories are recommended to have 755.

This issue could occur due to incorrect value of umask (recommended value 0022)

Fix: Try setting the permissions of all directories in path to 755 and try again.

Let us know how it goes.

avatar
Rising Star

i changed all jars permissions, and the result like below:

1123-4.png

then, i run the job again, but it still not works ok, and the error log as below:

1124-3.png

avatar

@Hefei Li

Can you set the directory permissions to 755? (all the directories ,including parents, in the path that contains the jar files)

avatar
Rising Star

@bsaini

yes, i am sure.

but i have a puzzle. try it like above, is need to do on each machine in the Hadoop cluster, or just do it on Oozie server machine.

avatar
Rising Star

@bsaini

Hi bsaini. I finally sloved it.

Like you said, is a permissions issue. All the jar files which installed in HDFS must be set to 755.

My HDP version is 2.3.x, and the default Oozie Share Library is hdfs://{my_host}:8020//user/oozie/share/lib/lib_20151119202305/oozie.

Thank you for your guidance.

avatar

@Hefei Li

Great! Can you accept the answer then so we can close this question and others having similar issue get benefited?