Support Questions

Find answers, ask questions, and share your expertise

HBASE OldWals not deleting

avatar
Explorer

Hello everybody,

My disk space is consuming more and more in HDFS and when I dig , I can see HBASE took 400GB of Old Wals .However If I look at Wals . it is almost empty . So Can I remove the complete OLDWals from the machine . Does it affect any data. I have read WALs are used only when there is crash in memory before writing it to disk . Still do not want to take the risk .

$ hdfs dfs -du -h /apps/hbase/data

  • 0/apps/hbase/data/.tmp ©37
  • 0/apps/hbase/data/MasterProcWALs
  • 12.6 K /apps/hbase/data/WALs ---- ONLY KBs
  • 7.5 G /apps/hbase/data/archive
  • 0/apps/hbase/data/corrupt
  • 52.9 T /apps/hbase/data/data
  • 42/apps/hbase/data/hbase.id
  • 7/apps/hbase/data/hbase.version
  • 379.5 G /apps/hbase/data/oldWALs -- Large data

My Question is if I set any parameter in Ambari to auto purge on regular interval and how can I check the replication is enabled or not . hbase.master.logcleaner.ttl where to set if in case i do not want to delete it .

1 REPLY 1

avatar
Super Collaborator

Hello @lenu 

 

If you have Replication enabled, WALs are likely to be persisted until the WALs are replicated. If you aren't using HBase Replication, Ensure there are no Peers (via "list_peers") & "hbase.replication" Property is false. If the oldWALs aren't removed, Enable TRACE Logging for the HBase Master Service, which would print the CleanerChore Thread removing or skipping any entries. 

 

- Smarak