Member since
11-05-2016
6
Posts
1
Kudos Received
0
Solutions
11-11-2016
12:30 AM
Thank you! Will watch.
... View more
11-07-2016
09:24 PM
The above resting state Read Latency in our system was cut by 98% to 192 ms by increasing hbase.client.scanner.max.result.size from a default of 2 MB to 50 MB. In our case, we have HBASE 'cell size' (==Maximum Record Size == hbase.client.keyvalue.maxsize) of 10MB which is 5x larger than said default. hbase.client.scanner.max.result.size can be used to change the default "chunk size" transited back to the client. Again, by default this value is 2 MB since HBase 1.1+. Note about hbase.server.scanner.max.result.size — This setting enforces a maximum result size (in bytes), when reached the server will return the results is has so far. This is a safety setting and should be kept large. The default is inifinite in 0.98 and 1.0.x and 100mb in 1.1 and later.
... View more
11-07-2016
07:57 PM
Yes, Apache Thrift really needs to emit a useful error msg in the above scenario.
... View more
11-07-2016
01:07 AM
I'm seeing a similar "Read Latency" of 10 seconds in Ambari but with HBASE at rest (i.e., nobody is running any scans or anything with it). This was observed after loading ~30 GB of data (HDFS 30% full now) into a three-node HBase cluster having two Region Servers (these RS's had finished auto balancing regions between them followed by me executing a manual compaction from HBase webGUI). Even restarting HBase has no effect on read latency. I noticed too that Grafana has (a corresponding?) graph : HBase-Performance --> Operation Latencies : Scan Next -- but oddly it's one of the very few graphs labeled, "No datapoints", as if it were unplugged. Read latency goes up to 32 seconds when running 'hbase shell scan tableX'... no selective filter used. I'm surprised Ambari doesn't have a built in alarm that trips on anything over 2 seconds (Ambari GUI reports 'no alarms' even when read latency is ~18 min!) Pretty sure we have a config problem but not sure what. I'm running all this in AWS/Ubuntu14 using Ambari 2.2.2.0 (HDP 2.4.2.0-258) managed HBASE 1.1.2.2.4.2.0-258.
... View more
11-06-2016
11:55 PM
Thank you Josh! That was it.
... View more
11-05-2016
12:34 AM
1 Kudo
In AWS/Ubuntu14 with Python 2.7.6, using Ambari 2.2.2.0 (HDP 2.4.2.0-258), which contains HBASE 1.1.2.2.4.2.0-258. Updating a python client developed for an earlier version of HBase (0.94.6.1). Trying to use Thrift 0.9.0 to gen py code -- neither hbase1.thrift or hbase2.thrift generate the expected 'gen-py' directories (or anything) for python: hadoop1:/usr/hdp/2.4.2.0-258/hbase/include/thrift$ thrift -verbose --gen py /usr/hdp/2.4.2.0-258/hbase/include/thrift/hbase2.thrift Scanning /usr/hdp/2.4.2.0-258/hbase/include/thrift/hbase2.thrift for includes Parsing /usr/hdp/2.4.2.0-258/hbase/include/thrift/hbase2.thrift for types Program: /usr/hdp/2.4.2.0-258/hbase/include/thrift/hbase2.thrift Generating "py" What am I doing wrong?
... View more
Labels: