question Re: How to delete data older than x days on hbase tables? in Archives of Support Questions (Read Only)

How to delete data older than x days on hbase tables?

YBSNR — Fri, 16 Sep 2022 09:24:27 GMT

Hi All,

Since my hadoop cluster capacity is low and there is no business need to keep old data, I'm trying to find and delete records older than 200 days in hbase tables. I found that there is no tool or ready to use program available to achieve this.

Can someone give me the best approach to accomplish this? Should I write a MR Job? If yes, is there any pseudo code or algorithm?

Thanks

Re: How to delete data older than x days on hbase tables?

Harsh J — Tue, 17 Mar 2015 07:20:34 GMT

You should be able to simply set a TTL on your tables and run a major
compaction to delete older-than-TTL-time data. More on TTL at
http://archive.cloudera.com/cdh5/cdh/5/hbase/book.html#ttl.

Re: How to delete data older than x days on hbase tables?

YBSNR — Tue, 17 Mar 2015 11:26:57 GMT

Thank you.. Looks like TTL is a good option. But I remember, Major compaction was running for days. When we keep the frequent/ periodic compaction enabled, regions were going offline. how to optimize and control the compactions? To enable TTL, should we compromize on the availability of region?

Please guide me

Re: How to delete data older than x days on hbase tables?

YBSNR — Mon, 23 Mar 2015 06:43:31 GMT

Hi Harsh,

The TTL option works well on most of the tables/cases. But, flume agents loads data to staging tables contineously. In this case, when we run compaction, the regions will go offline and data load fails. So, I had to turnoff the major compaction. Can you help me on how to handle major compaction on these tables to purge old data using TTL?

Thanks