Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hbase can't move data from disk after major_compact

Solved Go to solution
Highlighted

hbase can't move data from disk after major_compact

New Contributor

the flow variables value is defalt value. (hbase1.1.2 hdp2.3.4.0)

hbase.master.cleaner.interval

hbase.master.cleaner.interval

hbase.master.logcleaner.ttl

my hbase table is set 2 days ttl ,I do major_compact 'tablename' every day.

but 2 month later,I find hdfs /apps/hbase/data/data/default/tablename grows up every day.

why ? Please help me. Thanks very much

1 ACCEPTED SOLUTION

Accepted Solutions

Re: hbase can't move data from disk after major_compact

Super Guru
@Xiaojie Ma

One thing just popped up in my mind. Do you have snapshots that may be pointing to your data? In that case, data is not deleted by major compaction and moved to an archiving folder. See under /hbase if you have .archive folder.

9 REPLIES 9

Re: hbase can't move data from disk after major_compact

Super Guru

@Xiaojie Ma

Can you check the setting for keep deleted cells fo your column family?

https://hbase.apache.org/book.html#cf.keep.deleted

Also, have you set the following?

hbase.hstore.time.to.purge.deletes

If yes, what's the value?

Re: hbase can't move data from disk after major_compact

New Contributor

Both these two value which I didn't set,it may default values . But in my case, did these two values affect deletion ?

Re: hbase can't move data from disk after major_compact

New Contributor

hbase.hstore.time.to.purge.deletes default is 0

Re: hbase can't move data from disk after major_compact

New Contributor

I just want to move ttl timeout data away from my disk.

Re: hbase can't move data from disk after major_compact

New Contributor

hbase.hstore.time.to.purge.deletes has been changed; if it is not set, or set to 0, all delete markers including those with future timestamp are purged during the later major compaction. Otherwise, a delete marker is kept until the major compaction after marker's timestamp + this setting.

I think these two value which have nothing to do in my case.

Re: hbase can't move data from disk after major_compact

Super Guru

Can you please share the result of following command: describe 'TABLE_NAME'

Re: hbase can't move data from disk after major_compact

New Contributor

hbase(main):002:0> describe 'goods_new' Table goods_new is ENABLED goods_new COLUMN FAMILIES DESCRIPTION {NAME => 'v', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'TRUE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'LZO', MIN_VERSIONS => '0', BLOCKCACHE = > 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 1 row(s) in 0.2250 seconds hbase(main):003:0>

hbase api delete data very day.

Re: hbase can't move data from disk after major_compact

Super Guru
@Xiaojie Ma

One thing just popped up in my mind. Do you have snapshots that may be pointing to your data? In that case, data is not deleted by major compaction and moved to an archiving folder. See under /hbase if you have .archive folder.

Re: hbase can't move data from disk after major_compact

New Contributor

Thank you very much.

Don't have an account?
Coming from Hortonworks? Activate your account here