Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HBASE OldWals not deleting

avatar
Explorer

Hello everybody,

My disk space is consuming more and more in HDFS and when I dig , I can see HBASE took 400GB of Old Wals .However If I look at Wals . it is almost empty . So Can I remove the complete OLDWals from the machine . Does it affect any data. I have read WALs are used only when there is crash in memory before writing it to disk . Still do not want to take the risk .

$ hdfs dfs -du -h /apps/hbase/data

  • 0/apps/hbase/data/.tmp ©37
  • 0/apps/hbase/data/MasterProcWALs
  • 12.6 K /apps/hbase/data/WALs ---- ONLY KBs
  • 7.5 G /apps/hbase/data/archive
  • 0/apps/hbase/data/corrupt
  • 52.9 T /apps/hbase/data/data
  • 42/apps/hbase/data/hbase.id
  • 7/apps/hbase/data/hbase.version
  • 379.5 G /apps/hbase/data/oldWALs -- Large data

My Question is if I set any parameter in Ambari to auto purge on regular interval and how can I check the replication is enabled or not . hbase.master.logcleaner.ttl where to set if in case i do not want to delete it .

1 REPLY 1

avatar
Super Collaborator

Hello @lenu 

 

If you have Replication enabled, WALs are likely to be persisted until the WALs are replicated. If you aren't using HBase Replication, Ensure there are no Peers (via "list_peers") & "hbase.replication" Property is false. If the oldWALs aren't removed, Enable TRACE Logging for the HBase Master Service, which would print the CleanerChore Thread removing or skipping any entries. 

 

- Smarak