Support Questions
Find answers, ask questions, and share your expertise

DistCp Fail with error Requested replication 10 exceeds maximum 3

Hi

I have Ambari Version 2.5.2.0 with 4 Nodes. 1 master and 3 slave.

In HDFS config property I have set

Block replication = 3

dfs.replication.max = 3 (Default value was 50)

If I did not set dfs.replication.max to 3 I was getting some under replicated blocks. With dfs.replication.max = 3, there was no under replicated blocks.

Now the Issue Part.

I have another old hadoop system (hadoop 1.2.1) with 10 nodes. I want to copy the data from the old hadoop system to HDP using 'distcp'. But when I try to copy I keep on getting error as 'Requested replication 10 exceeds maximum 3'.

Why is this error generated when try to use distcp.

Please help solve this issue.

Regards,

Saurav Ranjit

1 REPLY 1

Explorer

please check the the value of mapreduce.client.submit.file.replication in mapred-site.xml. probably this is configured as 10. this value can be lowered to 3.