05-26-2018 05:18 AM - last edited on 05-27-2018 06:13 AM by cjervis
1) One of the map attempt task of very critical MR job (which runs every day) got failed four times -- out of 4 times first attempt got failed on one host(host120.xxxx.xxxx.xxx) after 2hours 4 min. but 2nd , 3rd and 4th attempt were failed on the same host(host203.xxxx.xxxx.xxx) . The execution time gap among of these 3 attempts like 3 sec.
host203.xxxx.xxxx.xxx having some build issues so 2nd time got failed but ideally 3rd attempt should be triggered in a different host rather than on same host.
Logs (2nd attempt)
WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: java.io.IOException: WritableName can't load class:
Caused by: java.io.IOException: WritableName can't load class: at org.apache.hadoop.io.WritableName.getClass(WritableName.java:77) at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:2029)
Please share your suggestions to fix this issue as this is very critical job.
2 ) Is there any way set the Map reducer configuration properties from CMD level rather than config files in Cloudera or code level or while submitting job .If so what is the command to set . Kindly share those details
06-02-2018 01:06 AM