question Re: Error Running MapReduce2 job using YARN in Archives of Support Questions (Read Only)

Error Running MapReduce2 job using YARN

vijaysl — Tue, 04 Oct 2016 02:37:07 GMT

We installed and configured HDP-2.5.0.0. When we run a mapreduce job it fails with the diagnostics message "We crashed durring a commit". The status on Map and Reduce tasks indicate that they were successful. Following is the error observed in syslog output. How do we troubleshoot this error?Any tips will be appreciated.

2016-10-03 15:19:20,017 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.io.IOException: Was asked to shut down.
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1559)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
2016-10-03 15:19:20,021 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1

Re: Error Running MapReduce2 job using YARN

chennuri_gouris — Tue, 04 Oct 2016 02:59:19 GMT

wrong mapreduce.jobhistory.address property's value. try to copy mapred-site.xml from the cluster

Re: Error Running MapReduce2 job using YARN

sshimpi — Tue, 04 Oct 2016 03:01:23 GMT

@Vijay Lakshman

Can you check the container size allocated ? Seems you are hitting memory issue.

Re: Error Running MapReduce2 job using YARN

vijaysl — Tue, 04 Oct 2016 23:08:06 GMT

Thank you for the suggestion. I have verified the hostname and port for mapreduce.jobhistory.address and they are accurate. We are using the copy of the config from the cluster.

Re: Error Running MapReduce2 job using YARN

vijaysl — Tue, 04 Oct 2016 23:09:21 GMT

Thank You for the suggestion. This appears to be plausible. Will allocate additional memory and post an update.

Re: Error Running MapReduce2 job using YARN

vijaysl — Wed, 02 Nov 2016 20:34:22 GMT

This issue was a result of misconfiguration of one of the nodes having an ip address conflict with another server. Once the ip address was corrected, we were able to review the logs and address classpath errors. Thanks everyone for their suggestions.

Re: Error Running MapReduce2 job using YARN

vnath87 — Wed, 26 Jul 2017 22:34:46 GMT

I faced similiar issue.

We had a HDP upgrade, still my jobs are pointing to the custom library (which are all of previous version). I have updated my custom library with latest jars. The issue is fixed.