Reply
Highlighted
Expert Contributor
Posts: 361
Registered: ‎01-25-2017

DistCp failure during commit

Hi,
 
DistCp failed during commit, i suspect from what i see on privillages, any insights?
 
2017-02-05 09:34:42,504 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:cloudera-scm (auth:SIMPLE)
cause:java.io.IOException: Was asked to shut down.
2017-02-05 09:34:42,504 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.io.IOException: Was asked to shut down.
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1546)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1540)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1473)
2017-02-05 09:34:42,506 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Posts: 642
Topics: 3
Kudos: 121
Solutions: 67
Registered: ‎08-16-2016

Re: DistCp failure during commit

It failed to even launch the AM for the job against YARN. There is a reconfiguration somewhere. Are you able to launch an MR job outside of distcip? From another hosts? Do other services work like HDFS from this host?
Expert Contributor
Posts: 361
Registered: ‎01-25-2017

Re: DistCp failure during commit

All others are working fine

Posts: 642
Topics: 3
Kudos: 121
Solutions: 67
Registered: ‎08-16-2016

Re: DistCp failure during commit

Can you share the yarn-site.xml and mapred-site.xml from the host that is running this distcp job?

Can you post the distcp command?
Expert Contributor
Posts: 361
Registered: ‎01-25-2017

Re: DistCp failure during commit

Even it failed on diiferent node? do you need specific definition from the xmls?

Expert Contributor
Posts: 361
Registered: ‎01-25-2017

Re: DistCp failure during commit

Any other ideas?

Announcements