How regionserver will utilize heap memory?

karthiknedunche — Thu, 29 Mar 2018 11:20:15 GMT

When i was planning how many region split need to be done initially to increase performance, i was confused about how region server utilizing heap memory, i have 128 GB RAM and 16 GB java heap memory, i am running both datanode and region server in the same machine

can anyone tell me

* How region server will use heap and what are the parameters will occupy heap memory

* Is region server will use only 16 GB heap or even other space of RAM

Re: How regionserver will utilize heap memory?

schhabra1 — Fri, 30 Mar 2018 19:21:51 GMT

---->How region server will use heap and what are the parameters will occupy heap memory

Region server heap memory consumption is mainly dependent on below three.

- Block cache (Buffer maintained in heap for read)

- Memstore Size (Buffer maintained in heap for write and flush)

- Other Objects created within region server while during various operations.

Below two are the parameter which controls the max % of heap block cache and block cache consume.

hfile.block.cache.size

hbase.regionserver.global.memstore.size

=>Below links will help in understading region server configuration for block cache/memstore and region server sizing.

https://hbase.apache.org/book.html#perf.rs.memstore.size

https://hbase.apache.org/book.html#ops.capacity.regions.count

https://hbase.apache.org/book.html#block.cache.usage

-----> Is region server will use only 16 GB heap or even other space of RAM

Region server will comsume maximum = 16GB of heap + XX:MaxDirectMemorySize which you have configured.

MaxDirectMemorySize—The JVM has a kind of memory called direct memory, which is distinct from normal JVM heap memory, that can run out. You can increase the direct buffer memory either by increasing the maximum heap size (see previous JVM Heap Size), which increases both the maximum heap and the maximum direct memory, or by only increasing the maximum direct memory using -XX:MaxDirectMemorySize. The following parameter added to the Java application startup increases the maximum direct memory size to 256 megabytes:

question How regionserver will utilize heap memory? in Archives of Support Questions (Read Only)

How regionserver will utilize heap memory?

Re: How regionserver will utilize heap memory?