hiveserver2 out of memory - what settings to use

[#1]

Our instance of hiveserver2 has been running out of memory about once per day requiring a restart. The machines running hive serve2 have 52gb of memory, but is there some kind of rule of thumb on how much memory to allocate, based on machine size??

ps ax | grep -i hiveserver2 returns:

hive 14668 1 0 Feb02 ? 00:02:16 /usr//bin/java -Xmx256m -Djava.net.preferIPv4Stack=true -Dhdp.version=2.3.4.7-4 -Dhdp.version=2.3.4.7-4 -Dhadoop.log.dir=/usr/hdp/2.3.4.7-4/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.3.4.7-4/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/2.3.4.7-4/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.7-4/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.3.4.7-4/hive/lib/hive-service-1.2.1.2.3.4.7-4.jar org.apache.hive.service.server.HiveServer2

[#2 · Accepted solution]

@Maxwell Flanders

Rule of thumb varies on your specific use case. How many users are connecting concurrently? Regardless 256 MB is very low.

Bump it to at least 4GB and see if this resolves your issue. Change -Xmx256m to -Xmx4096m. I was able to find recommended values for HS2. Use this as a guideline:

Up to 20 concurrent connections	6 GB
Up to 10 concurrent connections	4 GB
Single connection	2 GB

[#3]

That worked perfectly, thank you!