Member since
01-05-2017
5
Posts
0
Kudos Received
0
Solutions
03-14-2017
09:17 AM
This block size if specified in HColumnDescriptor. And HDFS block size default 64[MB]. Each row has 6[MB], So one block can keep 6rows. I think 6 is small..., Should I increase block size?
... View more
03-14-2017
06:48 AM
I checked log about "CompactSplitThread". It finished in several seconds~minutes and it treats 10~100 MB files before the total size reached around 1[TB]. But after then it treats several GB file and it takes several 10 minues. So I thought threshold is set in 1[TB] about heavy compaction/split. hbase.hregion.max.filesize causes "major compaction" as following site. But it was set 1[GB](default value). I do not think that this setting is the cause... https://jp.hortonworks.com/blog/apache-hbase-region-splitting-and-merging/ But I have no other choise, I increased this value to 10[TB] and change policy to ConstantSizeRegionSplitPolicy. I began to write record again. I hope this will not affect bad because my table is pre-splitted. I'll report if it goes well.
... View more
03-10-2017
06:36 AM
hbase-regionserver-node003-extract.zip Each node has one 7.2K rpm 3TB HDD. And today 2 nodes finished compaction. But compaction queue size of 1 node increased from yesterday night... I checked regionserver log. 3 type compaction(HStore, HRegion and CompactSplitThread) looks working every time. It takes serveral ten minutes and it treats several GB file for each time. I set pre-split to my table. But split looks working. Can it cause slowness? I attached my region server log file. Could you give me some advice if you have time? * I deleted "org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.gz]" from log file. Because it recorded several times every second.
... View more
03-09-2017
06:52 AM
Thank you for quick reply. Bulk import sounds good. I will try it. The reason of disabling major compaction is I was concerned about the load when majro and minor compaction were executed at the same time. And I'm preparing to getting aditional 3 PC. But it takes several month...
I think writing speed 0.5 [MB / s] with 3 nodes is slow. Is this speed normal? Or is tuning difficult in my small PC environment?
... View more
03-09-2017
05:25 AM
Hello. Now I'm evaluating to HBase performance and strugging with compaction. The queue of compaction does not increase until writing to 100[GB]. But after then, The queue continues to increase by writing data. The compaction queue size becomes around 3000 per each nodes when my app wrote 1.2[TB].
(Writing speed is 0.5[MB/s], and there is no reading operation while writing) The queue is decreasing by 1000 per day per node. It takes 3 days to store 1.2[TB], and it takes 3 days to completed compaction... My concern is
- I disabled major compaction. Does minor compaction takes such a long time? (or failed to disable it?)
- Can I shorten the compaction execution time with keeping writing speed?
- If I succeed in shortening, will it affect reading latency? I changed following settings from default.
- hbase.hregion.majorcompaction : 0
- hbase.hstore.blockingWaitTime : 0
- hbase.hregion.memstore.block.multiplier : 10
- Specifiy pre-split key when create table
- The block size is 1[MB] My HBase evaluation is
- region server : 3 nodes
- Software : windows, HDP-2.4.2, hbase-1.1.2.2.4.0.0-169
- Set 20[GB] memory to each region server
- Each row size is 6[MB]. column family number is 3, qualifier number is 18.
- The total row is 200K (so total data volume is 6[MB] * 200K = 1.2[TB])
- Writing performance : 0.5[MB/s]
... View more
Labels:
- Labels:
-
Apache HBase