Member since
03-22-2020
2
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
11130 | 03-25-2020 10:38 AM |
03-25-2020
10:38 AM
Below are the steps to troubleshoot distcp:- 1. it is not problem with the hdfs or Kerberos or distcp but a MapReduce. 2. We tried to run a sample MR job to test, then it failed with the following exception Error: Java.io.IOException: initialization of all the collectors failed. Error in last collector was:java.io.IOException: Invalid “mapreduce.task.io.sort.mb”:3276. (The total amount of buffer memory to use while sorting files, in MB). It was expecting less than 2048. Changing this property able to run the distcp smooth. I want to take a moment and say thanks to Shelton for responding it on time.
... View more