Reply
Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Deprecated HconnectionManager and "new" ConnectionFactory and ExecutorService

Hi, both these classes do some stuff inside to create HConnection/Connection.

I see that it's possible to provide executor service while creating HConnection/Connection.
 
1. Do I have to do that
2. looks like I have problems with performance because of shared connection. The more threads I use, the bigger response time I get. HBase is cold, it doesn't suffer because of load.
 
 
Here are metrics for my dummy perfomance test:
10 threads looks perfect in terms of respose: 99% <= 28.81 milliseconds
-- Meters ----------------------------------------------------------------------
.putMeter
count = 414914
mean rate = 885.58 events/second
1-minute rate = 911.56 events/second
5-minute rate = 778.16 events/second
15-minute rate = 549.72 events/second

-- Timers ----------------------------------------------------------------------
.putTimer
count = 414914
mean rate = 884.66 calls/second
1-minute rate = 911.53 calls/second
5-minute rate = 765.60 calls/second
15-minute rate = 515.06 calls/second
min = 4.87 milliseconds
max = 211.77 milliseconds
mean = 10.81 milliseconds
stddev = 5.43 milliseconds
median = 10.34 milliseconds
75% <= 11.59 milliseconds
95% <= 14.41 milliseconds
98% <= 19.59 milliseconds
99% <= 28.81 milliseconds
99.9% <= 60.67 milliseconds
 
50 threads give better throughput, but response is awful 99% <= 202.50 milliseconds
-- Meters ----------------------------------------------------------------------
.putMeter
count = 1348130
mean rate = 2217.72 events/second
1-minute rate = 2114.95 events/second
5-minute rate = 2194.61 events/second
15-minute rate = 1942.24 events/second

-- Timers ----------------------------------------------------------------------
.putTimer
count = 1348130
mean rate = 2216.04 calls/second
1-minute rate = 2120.26 calls/second
5-minute rate = 2169.94 calls/second
15-minute rate = 1844.06 calls/second
min = 8.01 milliseconds
max = 348.75 milliseconds
mean = 28.26 milliseconds
stddev = 31.91 milliseconds
median = 18.58 milliseconds
75% <= 27.48 milliseconds
95% <= 76.83 milliseconds
98% <= 100.05 milliseconds
99% <= 202.50 milliseconds
99.9% <= 348.75 milliseconds
100 threads give +10% throughput and the same response time.
 
 
What is the rgiht way to fix it?
Posts: 1,903
Kudos: 436
Solutions: 307
Registered: ‎07-31-2013

Re: Deprecated HconnectionManager and "new" ConnectionFactory and ExecutorService

Could you share your perf-test code snippet here, for us to apply and check what may be the backend issue thats causing you to observe these results?