question Re: Testing HBase bucket cache in Archives of Support Questions (Read Only)

Testing HBase bucket cache

joshua_adeleke — Mon, 01 Aug 2016 13:49:47 GMT

I am testing bucket cache implementation on my dev cluster (2 node) before implementing on the production cluster. On the dev, i have 32GB RAM and i have tried to configure bucket cache with the free memory but it always bring down the region server with OOM error. I need a configuration specification to test before i slam it on the production cluster (production cluster has excess of 100GB RAM). Find attached the hortonworks template i used and the error log.

Re: Testing HBase bucket cache

asinghal — Mon, 01 Aug 2016 14:09:24 GMT

can you please also attach hbase-site.xml and hbase-env.sh

Re: Testing HBase bucket cache

joshua_adeleke — Mon, 01 Aug 2016 14:30:38 GMT

Please find attached the hbase-site.xml and hbase-env.sh files. Thank you @Ankit Singhal.

Re: Testing HBase bucket cache

asinghal — Mon, 01 Aug 2016 15:36:38 GMT

As per your hbase-site.xml, bucket cache size is configured as

<property>
      <name>hbase.bucketcache.size</name>
      <value>6416</value>
    </property

But hbase-env.sh has MaxDirectMemorySize is set to just

-XX:MaxDirectMemorySize=6128

You need to set this -XX:MaxDirectMemorySize in hbase-env.sh to more than the bucket cache size configuration, and remember to suffix the configuration with "m"

-XX:MaxDirectMemorySize=6528m

Re: Testing HBase bucket cache

tyu — Mon, 01 Aug 2016 15:45:36 GMT

Which release of HDP are you using ?

Re: Testing HBase bucket cache

joshua_adeleke — Mon, 01 Aug 2016 15:53:27 GMT

Release 2.4.2

Re: Testing HBase bucket cache

joshua_adeleke — Wed, 10 Aug 2016 11:58:44 GMT

Thank you @Ankit Singhal. The implementation worked as advised.