Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Who agreed with this topic

Creating and deleting a table leads to java heap leak in hbase regionserver

Explorer

Hi,

 

I'm using CDH 5.9 (hbase-1.2.0+cdh5.9.0+205) and think I've found a heap leak when creating and deleting a table.

 

We're running an automated test that creates and deletes an hbase table every minute (and does other things too). We noticed that when running for a long time the regionserver java heap usage trends upwards at 190MB per day. The garbage collection never quite collects all the memory and the overall trend is up.

 

If I disable the part of the test that creates and deletes the table, and just uses one that is alreay created then the memory usage doesn't trend up any more.

 

If you look at the picture, you can see where I changed the test to stop the repeated table creation - where the up trend flattens out. And then the big drop off is where I restarted the regionserver.

 

The regionserver java heap contains an awful lot of counter objects:

 

1: 7770499 1056787864 org.apache.hadoop.hbase.util.Counter$Cell
2: 7770499 186491976 [Lorg.apache.hadoop.hbase.util.Counter$Cell;
3: 7770499 186491976 org.apache.hadoop.hbase.util.Counter$Container
4: 7770498 186491952 org.apache.hadoop.hbase.util.Counter
5: 7770498 186491952 org.apache.hadoop.hbase.util.Counter$1
6: 7814877 125038032 java.util.concurrent.atomic.AtomicBoolean
7: 7777721 124443536 java.util.concurrent.atomic.AtomicReference

 

Any hbase experts that could check this one out and confirm?

 

Cheers,

James

 

 

 

 


Screen Shot 2017-07-06 at 10.10.04.png

Who agreed with this topic