Member since
08-01-2013
4
Posts
6
Kudos Received
0
Solutions
08-14-2013
01:36 PM
This is how I invoke the mapreduce job hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming-2.0.0-cdh4.3.0.jar -input /user/certification/sandbox/access.log -output /user/certification/sandboxout -mapper /user/certification/sandbox/linecount_mapper.py -reducer /user/certification/sandbox/linecount_reduce.py I also tried using hadoop-streaming.jar, it gives me the same error as well. Any suggestions are greatly appreciated.
... View more
08-14-2013
01:12 PM
Hi, I have a similar problem. I wrote a simple mapper and reducer to read input file and calculate total number of lines. This works great locally cat access.log | ./linecount_mapper.py | ./linecount_reduce.py Same input files and scripts, when used in streaming returns this error message. Any suggestions? java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja Thanks, Charmee
... View more