Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Running Word Count using Hue oozie for java job is failing

Highlighted

Running Word Count using Hue oozie for java job is failing

New Contributor

Hi Friends ,

 

This is my first post in this community and I am learning, so kindly ignore my mistakes. I am trying to execute the word count problem using hue UI. I was able to run the same code using command prompt but now I am learning to execute through hue.

 

I did the configuration as Configuration Screen.JPGJob Configuration

My jar looks like this:

MyWordCount class.JPG

Now when I execute the job . I am not getting the desired result as it is showing the failed job. In the log I can see that it is unable to find the MyWordCount class. Below is the log:

 

 

Log Type: syslog
         
            Log Upload Time: Sat Feb 03 04:11:02 -0800 2018
         
            Log Length: 5721
          2018-02-03 04:09:59,617 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties
2018-02-03 04:10:00,537 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2018-02-03 04:10:00,538 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2018-02-03 04:10:00,627 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2018-02-03 04:10:00,627 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1517659378083_0003, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@12082570)
2018-02-03 04:10:01,617 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: RM_DELEGATION_TOKEN, Service: 127.0.0.1:8032, Ident: (RM_DELEGATION_TOKEN owner=cloudera, renewer=oozie mr token, realUser=oozie, issueDate=1517659727363, maxDate=1518264527363, sequenceNumber=6, masterKeyId=2)
2018-02-03 04:10:02,252 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2018-02-03 04:10:04,572 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/appcache/application_1517659378083_0003
2018-02-03 04:10:07,278 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2018-02-03 04:10:09,606 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2018-02-03 04:10:09,606 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2018-02-03 04:10:09,694 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2018-02-03 04:10:11,141 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: hdfs://quickstart.cloudera:8020/inputnew/inputFile.txt:60+60
2018-02-03 04:10:11,227 INFO [main] org.apache.hadoop.mapred.MapTask: numReduceTasks: 1
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2018-02-03 04:10:12,052 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2018-02-03 04:10:12,068 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object
 at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
 at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
 at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
 ... 9 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class MyWordCount not found
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2234)
 at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1114)
 at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
 ... 14 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class MyWordCount not found
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2202)
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2226)
 ... 16 more
Caused by: java.lang.ClassNotFoundException: Class MyWordCount not found
 at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2108)
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2200)
 ... 17 more

2018-02-03 04:10:12,092 INFO [main] org.apache.hadoop.mapred.Task: Runnning cleanup for the task
2018-02-03 04:10:12,105 WARN [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://quickstart.cloudera:8020/output_current_3rdFeb_2ndtrial/_temporary/1/_temporary/attempt_1517659378083_0003_m_000001_1
2018-02-03 04:10:12,116 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2018-02-03 04:10:12,117 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
2018-02-03 04:10:12,117 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.

 

 

Can you help me to know the issue why it is failing?