Reply
New Contributor
Posts: 1
Registered: ‎01-31-2018

Running Word Count using Hue oozie for java job is failing

[ Edited ]

Hi Friends ,

 

This is my first post in this community and I am learning, so kindly ignore my mistakes. I am trying to execute the word count problem using hue UI. I was able to run the same code using command prompt but now I am learning to execute through hue.

 

I did the configuration as Configuration Screen.JPGJob Configuration

My jar looks like this:

MyWordCount class.JPG

Now when I execute the job . I am not getting the desired result as it is showing the failed job. In the log I can see that it is unable to find the MyWordCount class. Below is the log:

 

 

Log Type: syslog
         
            Log Upload Time: Sat Feb 03 04:11:02 -0800 2018
         
            Log Length: 5721
          2018-02-03 04:09:59,617 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties
2018-02-03 04:10:00,537 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2018-02-03 04:10:00,538 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2018-02-03 04:10:00,627 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2018-02-03 04:10:00,627 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1517659378083_0003, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@12082570)
2018-02-03 04:10:01,617 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: RM_DELEGATION_TOKEN, Service: 127.0.0.1:8032, Ident: (RM_DELEGATION_TOKEN owner=cloudera, renewer=oozie mr token, realUser=oozie, issueDate=1517659727363, maxDate=1518264527363, sequenceNumber=6, masterKeyId=2)
2018-02-03 04:10:02,252 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2018-02-03 04:10:04,572 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/appcache/application_1517659378083_0003
2018-02-03 04:10:07,278 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2018-02-03 04:10:09,606 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2018-02-03 04:10:09,606 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2018-02-03 04:10:09,694 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2018-02-03 04:10:11,141 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: hdfs://quickstart.cloudera:8020/inputnew/inputFile.txt:60+60
2018-02-03 04:10:11,227 INFO [main] org.apache.hadoop.mapred.MapTask: numReduceTasks: 1
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 83886080
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600
2018-02-03 04:10:11,976 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600
2018-02-03 04:10:12,052 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2018-02-03 04:10:12,068 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object
 at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
 at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
 at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
 ... 9 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class MyWordCount not found
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2234)
 at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1114)
 at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
 ... 14 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class MyWordCount not found
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2202)
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2226)
 ... 16 more
Caused by: java.lang.ClassNotFoundException: Class MyWordCount not found
 at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2108)
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2200)
 ... 17 more

2018-02-03 04:10:12,092 INFO [main] org.apache.hadoop.mapred.Task: Runnning cleanup for the task
2018-02-03 04:10:12,105 WARN [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://quickstart.cloudera:8020/output_current_3rdFeb_2ndtrial/_temporary/1/_temporary/attempt_1517659378083_0003_m_000001_1
2018-02-03 04:10:12,116 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2018-02-03 04:10:12,117 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
2018-02-03 04:10:12,117 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.

 

 

Can you help me to know the issue why it is failing?

 

 

Announcements