OldWALs are not deleting

Question

Hello Team,

I am facing issue in my cloudera cluster and HDFS used space is growing and "/hbase/oldWALs" is occupied more than 50%.

I can confirm that, HBase replication is disabled and TTL is set to 1 Min.

hbase master logcleaner ttl = 1 Min

hbase replication = false

HBase logs i can see below warn's,

WARN org apache hadoop hbase master cleaner CleanerChore: A file cleanerhostname 60000.oldLogCleaner is stopped, won't delete any more files in /nameservice1/hbase/oldWALs

And checked list of peers in hbase,

hbase(main):001:0> list_peers
PEER_ID CLUSTER_KEY STATE TABLE_CFS
0 row(s) in 0.2360 seconds

I dont see anything, Please help me with your comments.

Thanks & Regards,

Vinod

smdas · Accepted Answer

Hello @kvinod

Thanks for using Cloudera Community. Your concern is HBase OldWALs on HDFS Path "/hbase/oldWALs" are occupying a lot of space. HBase Replication isn't being used & TTL is set to 1 Minute.

The HMaster Trace Logs capture the CleanerChore with verbose logging, yet I wish to check if you have tried the following 2 options:

1. Restart the HMaster Service to confirm if any issues with CleanerChore,

2. The Parameter "hbase.replication" is set correctly to False via the Steps shared under Section [1].

3. If the "/hbase/replication" has any entries. If no Replication is utilised (HBase Replication or Lily Indexer), Try removing the "/hbase/replication" ZNode & restart the HMaster Service.

- Smarak

[1] CM=> HBase=> Configuration=> Advanced=> HBase Service Advanced Configuration Snippet (Safety Valve) for hbase-site.xml

kvinod · Answer

Can someone please help me?That would be great and thanks in advance...!!

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.