Reply
Highlighted
Explorer
Posts: 19
Registered: ‎06-23-2014

Changing Max Block Replication from 4 to 3 results in "Requested replication 4 exceeds max 3" error

I have a cluster in with the following parameters:

 

Replication Factor (dfs.replication) is set to "2"
Minimal Block Replication (dfs.replication.min, dfs.namenode.replication.min) is set to "2"
Maximal Block Replication (dfs.replication.max) is set to "4"
 
I changed the Maximal Block Replication to "3" and restarted HDFS
 
Now MapReduce jobs fails with "Requested replication 4 exceeds maximum 3 at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.verifyReplication"
 
Why? Shouldn't this new value be used as the max? Why the system attempting to use 4 as the replication number?

 

Cloudera Employee
Posts: 314
Registered: ‎01-16-2014

Re: Changing Max Block Replication from 4 to 3 results in "Requested replication 4 exceeds max

Check the mapreduce.client.submit.file.replication you have set under the YARN/MR config.

That could be the issue for any files added during the job submission.

 

Wilfred

Announcements