Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS data is not decreasing after reducing the replication from 3 to 2

Solved Go to solution
Highlighted

HDFS data is not decreasing after reducing the replication from 3 to 2

 
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Mentor

@ram sriram

The below command sets your replication factor to 1 for all new files you will create, with a potential data loss unless you are running HDP 3.x which has a new HDFS algorithm EC erasure coding

$ hdfs dfs -setrep -w 1 -R /

As responded above the changes only affect new files you will create. After changing the replication factor you won't see any hdfs size changes until the trash time interval which was set on 360 minutes configured by the hdfs trash interval has been reached

fs.trash.interval 

Once the NameNode metadata has been updated, it is the DataNodes which would actually do the operation. There could be some delay, but space is definitely reclaimed.

HTH


View solution in original post

6 REPLIES 6
Highlighted

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

@ram sriram

Changing replication will only apply to new files you need to run this Command to change replication factor for files that already exist

hdfs dfs -setrep -w 1 -R /

Highlighted

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

@Hariprasanth Madhavan

Thanks for u r reply.i ran that command but still its showing size.

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Check your snapshot details

hdfs dfs -ls $DIR/.snapshot/

hdfs dfs -setrep -w 2 -R / then restart hdfs services

Highlighted

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Mentor

@ram sriram

The below command sets your replication factor to 1 for all new files you will create, with a potential data loss unless you are running HDP 3.x which has a new HDFS algorithm EC erasure coding

$ hdfs dfs -setrep -w 1 -R /

As responded above the changes only affect new files you will create. After changing the replication factor you won't see any hdfs size changes until the trash time interval which was set on 360 minutes configured by the hdfs trash interval has been reached

fs.trash.interval 

Once the NameNode metadata has been updated, it is the DataNodes which would actually do the operation. There could be some delay, but space is definitely reclaimed.

HTH


View solution in original post

Highlighted

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Hi @Geoffrey Shelton Okot Thank you for your valuable reply.

Can you please send me a document for Ambari installation on ubuntu.

Highlighted

Re: HDFS data is not decreasing after reducing the replication from 3 to 2

Mentor

@ram sriram

If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer.

Can you tag me for "Can you please send me a document for Ambari installation on ubuntu thread" so I see the information you already received.

Don't have an account?
Coming from Hortonworks? Activate your account here