Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!
Labels (1)
avatar
Expert Contributor

By Default, HiveServer2 and HiveMetastore does not have configuration for HeapDump on OOM. Production clusters have OOM and since the HeapDump on OOM is not configured, root cause analysis of the issue is obstructed.

Navigate as below in Ambari:

Ambari UI > Hive > Configs > Advanced hive-env > hive-env template

Add following :

if [ "$SERVICE" = "metastore" ]; then
  export HADOOP_HEAPSIZE={{hive_metastore_heapsize}} # Setting for HiveMetastore
else
  export HADOOP_HEAPSIZE={{hive_heapsize}} # Setting for HiveServer2 and Client
fi

export HADOOP_CLIENT_OPTS="-Xmx${HADOOP_HEAPSIZE}m -Xloggc:/var/log/hive/gc.log-$SERVICE-`date +'%Y%m%d%H%M'`
-XX:ErrorFile=/var/log/hive/hive-metastore-error.log-`date +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps  -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/hive/
$HADOOP_CLIENT_OPTS"

if [ "$SERVICE" = "hiveserver2" ]; then
export HADOOP_CLIENT_OPTS="-Xmx${HADOOP_HEAPSIZE}m -Xloggc:/var/log/hive/gc.log-$SERVICE-`date +'%Y%m%d%H%M'`
-XX:ErrorFile=/var/log/hive/hive-server2-error.log-`date +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/hive/  -XX:+PrintGCDateStamps $HADOOP_CLIENT_OPTS"
fi
2,585 Views
0 Kudos
Comments

The script seems wrong.
With this configuration, the lines for "hiveserver2" will never be valid. The settings in the previous lines "export HADOOP_CLIENT_OPTS=" will overwirte them all with the "$HADOOP_CLIENT_OPTS" at the end. The name of ErrorFile will become "hive-metastore-error.log-`date +'%Y%m%d%H%M'" for example.
The script should be:

if [ "$SERVICE" = "metastore" ]; then
....
elif [ "$SERVICE" = "hiveserver2" ]; then
....
fi

Version history
Last update:
‎09-21-2017 03:31 PM
Updated by:
Contributors