Support Questions
Find answers, ask questions, and share your expertise

Hive garbage collector best practice

Hive garbage collector best practice

Rising Star

Hi,

I see an increase of Old Generation memory from jmap for Hiverser2.

This is my configuration:

/usr/jdk64/jdk1.8.0_40/bin/java -Xmx2048m -Dhdp.version=2.3.2.0-2950 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.3.2.0-2950 -Dhadoop.log.dir=/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.3.2.0-2950/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.3.2.0-2950/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx2048m -Xmx8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/hive/oom/ -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:ErrorFile=/var/log/hive/oom/hs2jvmerror%p.log -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.3.2.0-2950/hive/lib/hive-service-1.2.1.2.3.2.0-2950.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris= -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/var/log/hive

jmap -heap output is:

JVM version is 25.40-b25

using thread-local object allocation.

Parallel GC with 8 thread(s)

Heap Configuration:

MinHeapFreeRatio = 0

MaxHeapFreeRatio = 100

MaxHeapSize = 8589934592 (8192.0MB)

NewSize = 335544320 (320.0MB)

MaxNewSize = 2863136768 (2730.5MB)

OldSize = 671088640 (640.0MB)

NewRatio = 2

SurvivorRatio = 8

MetaspaceSize = 21807104 (20.796875MB)

CompressedClassSpaceSize = 1073741824 (1024.0MB)

MaxMetaspaceSize = 17592186044415 MB

G1HeapRegionSize = 0 (0.0MB)

Heap Usage:

PS Young Generation

Eden Space: capacity = 354418688 (338.0MB)

used = 70879872 (67.5963134765625MB)

free = 283538816 (270.4036865234375MB)

19.998909312592456% used

From Space:

capacity = 14680064 (14.0MB)

used = 14244056 (13.584190368652344MB)

free = 436008 (0.41580963134765625MB)

97.0299312046596% used

To Space:

capacity = 67108864 (64.0MB)

used = 0 (0.0MB)

free = 67108864 (64.0MB)

0.0% used

PS Old Generation

capacity = 3139960832 (2994.5MB)

used = 2938877792 (2802.7322692871094MB)

free = 201083040 (191.76773071289062MB)

93.59600164592116% used

39810 interned Strings occupying 4013432 bytes.

Do you have any best practice for tuning this?

1 REPLY 1
Highlighted

Re: Hive garbage collector best practice

Why are you asking if I may ask? Do you see any performance problems because of GC pauses? Is your hive server running into GC death sometimes? If that is the case you might have an older version of HiveServer with a memory leak. That happened a couple times independently of each other. Contacting support and getting a fix might help in this case. ( Removing the ATSHooks can mitigate some of them )

If on the other hand a full compaction removes most of the old generation then we might think about tuning generation sizes. here is what I got from the hive dev team last time but that was before G1GC

-XX:NewRatio=12 -Xmx2048m
-Xms8192m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15
Don't have an account?