Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

hbase can't move data from disk after major_compact

avatar
Contributor

the flow variables value is defalt value. (hbase1.1.2 hdp2.3.4.0)

hbase.master.cleaner.interval

hbase.master.cleaner.interval

hbase.master.logcleaner.ttl

my hbase table is set 2 days ttl ,I do major_compact 'tablename' every day.

but 2 month later,I find hdfs /apps/hbase/data/data/default/tablename grows up every day.

why ? Please help me. Thanks very much

1 ACCEPTED SOLUTION

avatar
Super Guru
@Xiaojie Ma

One thing just popped up in my mind. Do you have snapshots that may be pointing to your data? In that case, data is not deleted by major compaction and moved to an archiving folder. See under /hbase if you have .archive folder.

View solution in original post

9 REPLIES 9

avatar
Super Guru

@Xiaojie Ma

Can you check the setting for keep deleted cells fo your column family?

https://hbase.apache.org/book.html#cf.keep.deleted

Also, have you set the following?

hbase.hstore.time.to.purge.deletes

If yes, what's the value?

avatar
Contributor

Both these two value which I didn't set,it may default values . But in my case, did these two values affect deletion ?

avatar
Contributor

hbase.hstore.time.to.purge.deletes default is 0

avatar
Contributor

I just want to move ttl timeout data away from my disk.

avatar
Contributor

hbase.hstore.time.to.purge.deletes has been changed; if it is not set, or set to 0, all delete markers including those with future timestamp are purged during the later major compaction. Otherwise, a delete marker is kept until the major compaction after marker's timestamp + this setting.

I think these two value which have nothing to do in my case.

avatar
Super Guru

Can you please share the result of following command: describe 'TABLE_NAME'

avatar
Contributor

hbase(main):002:0> describe 'goods_new' Table goods_new is ENABLED goods_new COLUMN FAMILIES DESCRIPTION {NAME => 'v', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'TRUE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'LZO', MIN_VERSIONS => '0', BLOCKCACHE = > 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 1 row(s) in 0.2250 seconds hbase(main):003:0>

hbase api delete data very day.

avatar
Super Guru
@Xiaojie Ma

One thing just popped up in my mind. Do you have snapshots that may be pointing to your data? In that case, data is not deleted by major compaction and moved to an archiving folder. See under /hbase if you have .archive folder.

avatar
Contributor

Thank you very much.