Support Questions
Find answers, ask questions, and share your expertise

HDFS data is not decreasing after reducing the replication from 3 to 2

 
1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Mentor

@ram sriram

The below command sets your replication factor to 1 for all new files you will create, with a potential data loss unless you are running HDP 3.x which has a new HDFS algorithm EC erasure coding

$ hdfs dfs -setrep -w 1 -R /

As responded above the changes only affect new files you will create. After changing the replication factor you won't see any hdfs size changes until the trash time interval which was set on 360 minutes configured by the hdfs trash interval has been reached

fs.trash.interval 

Once the NameNode metadata has been updated, it is the DataNodes which would actually do the operation. There could be some delay, but space is definitely reclaimed.

HTH


View solution in original post

6 REPLIES 6

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

@ram sriram

Changing replication will only apply to new files you need to run this Command to change replication factor for files that already exist

hdfs dfs -setrep -w 1 -R /

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

@Hariprasanth Madhavan

Thanks for u r reply.i ran that command but still its showing size.

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Check your snapshot details

hdfs dfs -ls $DIR/.snapshot/

hdfs dfs -setrep -w 2 -R / then restart hdfs services

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Mentor

@ram sriram

The below command sets your replication factor to 1 for all new files you will create, with a potential data loss unless you are running HDP 3.x which has a new HDFS algorithm EC erasure coding

$ hdfs dfs -setrep -w 1 -R /

As responded above the changes only affect new files you will create. After changing the replication factor you won't see any hdfs size changes until the trash time interval which was set on 360 minutes configured by the hdfs trash interval has been reached

fs.trash.interval 

Once the NameNode metadata has been updated, it is the DataNodes which would actually do the operation. There could be some delay, but space is definitely reclaimed.

HTH


View solution in original post

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Hi @Geoffrey Shelton Okot Thank you for your valuable reply.

Can you please send me a document for Ambari installation on ubuntu.

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Mentor

@ram sriram

If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer.

Can you tag me for "Can you please send me a document for Ambari installation on ubuntu thread" so I see the information you already received.