Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Deprecated HconnectionManager and "new" ConnectionFactory and ExecutorService

Deprecated HconnectionManager and "new" ConnectionFactory and ExecutorService

Expert Contributor

Hi, both these classes do some stuff inside to create HConnection/Connection.

I see that it's possible to provide executor service while creating HConnection/Connection.
 
1. Do I have to do that
2. looks like I have problems with performance because of shared connection. The more threads I use, the bigger response time I get. HBase is cold, it doesn't suffer because of load.
 
 
Here are metrics for my dummy perfomance test:
10 threads looks perfect in terms of respose: 99% <= 28.81 milliseconds
-- Meters ----------------------------------------------------------------------
.putMeter
count = 414914
mean rate = 885.58 events/second
1-minute rate = 911.56 events/second
5-minute rate = 778.16 events/second
15-minute rate = 549.72 events/second

-- Timers ----------------------------------------------------------------------
.putTimer
count = 414914
mean rate = 884.66 calls/second
1-minute rate = 911.53 calls/second
5-minute rate = 765.60 calls/second
15-minute rate = 515.06 calls/second
min = 4.87 milliseconds
max = 211.77 milliseconds
mean = 10.81 milliseconds
stddev = 5.43 milliseconds
median = 10.34 milliseconds
75% <= 11.59 milliseconds
95% <= 14.41 milliseconds
98% <= 19.59 milliseconds
99% <= 28.81 milliseconds
99.9% <= 60.67 milliseconds
 
50 threads give better throughput, but response is awful 99% <= 202.50 milliseconds
-- Meters ----------------------------------------------------------------------
.putMeter
count = 1348130
mean rate = 2217.72 events/second
1-minute rate = 2114.95 events/second
5-minute rate = 2194.61 events/second
15-minute rate = 1942.24 events/second

-- Timers ----------------------------------------------------------------------
.putTimer
count = 1348130
mean rate = 2216.04 calls/second
1-minute rate = 2120.26 calls/second
5-minute rate = 2169.94 calls/second
15-minute rate = 1844.06 calls/second
min = 8.01 milliseconds
max = 348.75 milliseconds
mean = 28.26 milliseconds
stddev = 31.91 milliseconds
median = 18.58 milliseconds
75% <= 27.48 milliseconds
95% <= 76.83 milliseconds
98% <= 100.05 milliseconds
99% <= 202.50 milliseconds
99.9% <= 348.75 milliseconds
100 threads give +10% throughput and the same response time.
 
 
What is the rgiht way to fix it?
1 REPLY 1

Re: Deprecated HconnectionManager and "new" ConnectionFactory and ExecutorService

Master Guru
Could you share your perf-test code snippet here, for us to apply and check what may be the backend issue thats causing you to observe these results?
Don't have an account?
Coming from Hortonworks? Activate your account here