Below are the steps to troubleshoot distcp:- 1. it is not problem with the hdfs or Kerberos or distcp but a MapReduce. 2. We tried to run a sample MR job to test, then it failed with the following exception Error: Java.io.IOException: initialization of all the collectors failed. Error in last collector was:java.io.IOException: Invalid “mapreduce.task.io.sort.mb”:3276. (The total amount of buffer memory to use while sorting files, in MB). It was expecting less than 2048. Changing this property able to run the distcp smooth. I want to take a moment and say thanks to Shelton for responding it on time.
... View more
