Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Changing Max Block Replication from 4 to 3 results in "Requested replication 4 exceeds max 3" error

Highlighted

Changing Max Block Replication from 4 to 3 results in "Requested replication 4 exceeds max 3" error

I have a cluster in with the following parameters:

 

Replication Factor (dfs.replication) is set to "2"
Minimal Block Replication (dfs.replication.min, dfs.namenode.replication.min) is set to "2"
Maximal Block Replication (dfs.replication.max) is set to "4"
 
I changed the Maximal Block Replication to "3" and restarted HDFS
 
Now MapReduce jobs fails with "Requested replication 4 exceeds maximum 3 at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.verifyReplication"
 
Why? Shouldn't this new value be used as the max? Why the system attempting to use 4 as the replication number?

 

1 REPLY 1

Re: Changing Max Block Replication from 4 to 3 results in "Requested replication 4 exceeds max

Super Collaborator

Check the mapreduce.client.submit.file.replication you have set under the YARN/MR config.

That could be the issue for any files added during the job submission.

 

Wilfred

Don't have an account?
Coming from Hortonworks? Activate your account here