Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDFS data is not decreasing after reducing the replication from 3 to 2

avatar
 
1 ACCEPTED SOLUTION

avatar
Master Mentor

@ram sriram

The below command sets your replication factor to 1 for all new files you will create, with a potential data loss unless you are running HDP 3.x which has a new HDFS algorithm EC erasure coding

$ hdfs dfs -setrep -w 1 -R /

As responded above the changes only affect new files you will create. After changing the replication factor you won't see any hdfs size changes until the trash time interval which was set on 360 minutes configured by the hdfs trash interval has been reached

fs.trash.interval 

Once the NameNode metadata has been updated, it is the DataNodes which would actually do the operation. There could be some delay, but space is definitely reclaimed.

HTH


View solution in original post

6 REPLIES 6

avatar

@ram sriram

Changing replication will only apply to new files you need to run this Command to change replication factor for files that already exist

hdfs dfs -setrep -w 1 -R /

avatar

@Hariprasanth Madhavan

Thanks for u r reply.i ran that command but still its showing size.

avatar

Check your snapshot details

hdfs dfs -ls $DIR/.snapshot/

hdfs dfs -setrep -w 2 -R / then restart hdfs services

avatar
Master Mentor

@ram sriram

The below command sets your replication factor to 1 for all new files you will create, with a potential data loss unless you are running HDP 3.x which has a new HDFS algorithm EC erasure coding

$ hdfs dfs -setrep -w 1 -R /

As responded above the changes only affect new files you will create. After changing the replication factor you won't see any hdfs size changes until the trash time interval which was set on 360 minutes configured by the hdfs trash interval has been reached

fs.trash.interval 

Once the NameNode metadata has been updated, it is the DataNodes which would actually do the operation. There could be some delay, but space is definitely reclaimed.

HTH


avatar

Hi @Geoffrey Shelton Okot Thank you for your valuable reply.

Can you please send me a document for Ambari installation on ubuntu.

avatar
Master Mentor

@ram sriram

If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer.

Can you tag me for "Can you please send me a document for Ambari installation on ubuntu thread" so I see the information you already received.