Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Testing HBase bucket cache

avatar
Expert Contributor

I am testing bucket cache implementation on my dev cluster (2 node) before implementing on the production cluster. On the dev, i have 32GB RAM and i have tried to configure bucket cache with the free memory but it always bring down the region server with OOM error. I need a configuration specification to test before i slam it on the production cluster (production cluster has excess of 100GB RAM). Find attached the hortonworks template i used and the error log.

1 ACCEPTED SOLUTION

avatar

As per your hbase-site.xml, bucket cache size is configured as

<property>
      <name>hbase.bucketcache.size</name>
      <value>6416</value>
    </property
But hbase-env.sh has MaxDirectMemorySize is set to just  
-XX:MaxDirectMemorySize=6128

You need to set this -XX:MaxDirectMemorySize in hbase-env.sh to more than the bucket cache size configuration, and remember to suffix the configuration with "m"

-XX:MaxDirectMemorySize=6528m

View solution in original post

6 REPLIES 6

avatar

can you please also attach hbase-site.xml and hbase-env.sh

avatar
Expert Contributor

Please find attached the hbase-site.xml and hbase-env.sh files. Thank you @Ankit Singhal.

avatar

As per your hbase-site.xml, bucket cache size is configured as

<property>
      <name>hbase.bucketcache.size</name>
      <value>6416</value>
    </property
But hbase-env.sh has MaxDirectMemorySize is set to just  
-XX:MaxDirectMemorySize=6128

You need to set this -XX:MaxDirectMemorySize in hbase-env.sh to more than the bucket cache size configuration, and remember to suffix the configuration with "m"

-XX:MaxDirectMemorySize=6528m

avatar
Expert Contributor

Thank you @Ankit Singhal. The implementation worked as advised.

avatar
Master Collaborator

Which release of HDP are you using ?

avatar
Expert Contributor

Release 2.4.2