About sunny11541

raus · ‎04-15-2020

I am using ConstantSizeRegionSplitPolicy and MaxFileSize is set to 30 GB. But, I found that file is not split across regions when file size reaches 30 GB. Some of my file size is 300 GB across particular regions. Can you please help me to solve this probelm. I have huge volume of data 10 TB.

vrodionov · ‎08-19-2016

Yes, with IncreasingToUpeerBoundRegionSplitPolicy it is possible to have a split of a region which is far from maximum size - this is expected behavior. The reason why? HBase tries to create many regions while they are small and distribute them across the cluster. You will need to switch to ConstantSizeRegionSplitPolicy if you do not want this. hbase.regionserver.region.split.policy controls the setting per HBase table.

elserj · ‎07-17-2016

I would also say that you should be able to understand the data which you are loading to make sure that you are creating reasonable split points. Even if the keys are hashed, you should be able to understand what the first byte/character of the rowKey is and create reasonable split points (using RegionSplitter or by hand).

Online	Offline
Last Visited	‎11-30-2017 02:17 AM

Member Since	‎07-13-2016 01:18 AM
Last Visited	‎11-30-2017 02:17 AM
Posts	7
Kudos received	2

Cloudera Community

Re: Number of regions increases after major compac...

Re: Hbase region split

Re: Pre-splitting Hbase table not working