question Re: HDFS underreplicated blocks in Archives of Support Questions (Read Only)

HDFS underreplicated blocks

ahaeni — Tue, 30 May 2017 06:18:30 GMT

Hello,

I have installed a new cluster with CM 5.11 and Hadoop 2.6.0-cdh5.11.0. I have 1 Name Node with all other services running on it, and 2 Datanodes. HDFS Replication factor is set to 2.

I get the following HDFS error: Bad : 666 under replicated blocks in the cluster. 675 total blocks in the cluster. Percentage under replicated blocks: 98.67%. Critical threshold: 40.00%.

I also tried setting the replication factor to 1 to test it out and restarted the cluster and agents, but the issue remains unchanged. There are a few threads about this issue already but I haven't found a solution yet. Do you have any idea? Thanks in advance!

Re: HDFS underreplicated blocks

ahaeni — Tue, 30 May 2017 14:48:33 GMT

I fixed the issue.

The problem was that after setting the replication factor in Cloudera Manager files which were already created still preserve the replication factor they were created with (in my case 3). I had to recursively set the factor to 2 on the command line:

hdfs dfs -setrep -R 2 /*

Maybe it will help someone.

Re: HDFS underreplicated blocks

Soundharya — Sat, 03 Jun 2023 08:45:00 GMT

Very helpful .... Solved my problem! Thank you so much