Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Testing HBase bucket cache

Solved Go to solution
Highlighted

Testing HBase bucket cache

Expert Contributor

I am testing bucket cache implementation on my dev cluster (2 node) before implementing on the production cluster. On the dev, i have 32GB RAM and i have tried to configure bucket cache with the free memory but it always bring down the region server with OOM error. I need a configuration specification to test before i slam it on the production cluster (production cluster has excess of 100GB RAM). Find attached the hortonworks template i used and the error log.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Testing HBase bucket cache

As per your hbase-site.xml, bucket cache size is configured as

<property>
      <name>hbase.bucketcache.size</name>
      <value>6416</value>
    </property
But hbase-env.sh has MaxDirectMemorySize is set to just  
-XX:MaxDirectMemorySize=6128

You need to set this -XX:MaxDirectMemorySize in hbase-env.sh to more than the bucket cache size configuration, and remember to suffix the configuration with "m"

-XX:MaxDirectMemorySize=6528m

View solution in original post

6 REPLIES 6
Highlighted

Re: Testing HBase bucket cache

can you please also attach hbase-site.xml and hbase-env.sh

Highlighted

Re: Testing HBase bucket cache

Expert Contributor

Please find attached the hbase-site.xml and hbase-env.sh files. Thank you @Ankit Singhal.

Highlighted

Re: Testing HBase bucket cache

As per your hbase-site.xml, bucket cache size is configured as

<property>
      <name>hbase.bucketcache.size</name>
      <value>6416</value>
    </property
But hbase-env.sh has MaxDirectMemorySize is set to just  
-XX:MaxDirectMemorySize=6128

You need to set this -XX:MaxDirectMemorySize in hbase-env.sh to more than the bucket cache size configuration, and remember to suffix the configuration with "m"

-XX:MaxDirectMemorySize=6528m

View solution in original post

Highlighted

Re: Testing HBase bucket cache

Expert Contributor

Thank you @Ankit Singhal. The implementation worked as advised.

Highlighted

Re: Testing HBase bucket cache

Super Collaborator

Which release of HDP are you using ?

Highlighted

Re: Testing HBase bucket cache

Expert Contributor

Release 2.4.2

Don't have an account?
Coming from Hortonworks? Activate your account here