Support Questions

silvio_delval · ‎06-20-2016

Hello,

I have been looking for some information about this, but I did not found anything. Our HDFS file system is growing a lot, and I can see about 60% of space is in "oldWALs" folder. Why is it growing so much? How can I get rid of this?

We deployed our cluster with Ambari with almost no customization. We don't have replication to other clusters or so on...everything quite standard.

Best regards,

Silvio

tyu · ‎06-20-2016

There are currently two services which may keep the files in the archive directory. First is a TTL process, which ensures that the WAL files are kept at least for 10 min. This is controlled by hbase.master.logcleaner.ttl configuration property in master.

The other one is replication. If you had replication setup before, the replication processes will hang on to the WAL files until they are replicated. Even if you disabled the replication, the files are still referenced.

You can look at the logs of master from classes (LogCleaner, TimeToLiveLogCleaner, ReplicationLogCleaner) to see whether any exception was thrown.

View solution in original post

sanketplus · ‎04-17-2017

I am in exact same situation as you were in. Adding replaction=false property did not help and it's still growing. and strange part is it's been growing for last 7 days only. It was normal before that. Any clues what's happening here?

sanketplus · ‎04-18-2017

https://community.hortonworks.com/questions/97184/oldwals-not-getting-cleared-even-with-no-replicati...

Cloudera Community

Support Questions

oldWALs folder growing too much....