question Re: How to enable flush per column family instead of Region full flush? in Archives of Support Questions (Read Only)

How to enable flush per column family instead of Region full flush?

sunile_manjee — Mon, 23 May 2016 22:11:56 GMT

HBASE-10201 enabled logic to flush based on size of CF. If the size of CF > DEFAULT_HREGION_MEMSTORE_PER_COLUMN_FAMILY_FLUSH then it will flush those CFs and not entire memstore. How to enable this? I don't see much documentation on this feature.

Re: How to enable flush per column family instead of Region full flush?

asinghal — Mon, 23 May 2016 22:19:29 GMT

It is enabled by default(FlushLargeStoresPolicy). You just need to configure below property. It will not flush those column family who size is less than the size specified in below property

hbase.hregion.percolumnfamilyflush.size.lower.bound

Re: How to enable flush per column family instead of Region full flush?

tyu — Mon, 23 May 2016 22:55:07 GMT

There is also hbase.hregion.percolumnfamilyflush.size.lower.bound.min :

If FlushLargeStoresPolicy is used and there are multiple column families, then every time that we hit the total memstore limit, we find out all the column families whose memstores exceed a "lower bound" and only flush them while retaining the others in memory. The "lower bound" will be "hbase.hregion.memstore.flush.size / column_family_number" by default unless value of this property is larger than that. If none of the families have their memstore size more than lower bound, all the memstores will be flushed (just as usual).

Default:

16777216

Re: How to enable flush per column family instead of Region full flush?

tyu — Tue, 24 May 2016 09:37:15 GMT

Note: hbase.hregion.percolumnfamilyflush.size.lower.bound is used in HDP 2.3 / 2.4

hbase.hregion.percolumnfamilyflush.size.lower.bound.min would be used in HDP 2.5