Hello everybody,
My disk space is consuming more and more in HDFS and when I dig , I can see HBASE took 400GB of Old Wals .However If I look at Wals . it is almost empty . So Can I remove the complete OLDWals from the machine . Does it affect any data. I have read WALs are used only when there is crash in memory before writing it to disk . Still do not want to take the risk .
$ hdfs dfs -du -h /apps/hbase/data
- 0/apps/hbase/data/.tmp ©37
- 0/apps/hbase/data/MasterProcWALs
- 12.6 K /apps/hbase/data/WALs ---- ONLY KBs
- 7.5 G /apps/hbase/data/archive
- 0/apps/hbase/data/corrupt
- 52.9 T /apps/hbase/data/data
- 42/apps/hbase/data/hbase.id
- 7/apps/hbase/data/hbase.version
- 379.5 G /apps/hbase/data/oldWALs -- Large data
My Question is if I set any parameter in Ambari to auto purge on regular interval and how can I check the replication is enabled or not . hbase.master.logcleaner.ttl where to set if in case i do not want to delete it .