<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>question Re: Yarn containers get KILLED automatically. MapReduce/Hive jobs fail in Support Questions</title>
    <link>https://community.cloudera.com/t5/Support-Questions/Yarn-containers-get-KILLED-automatically-MapReduce-Hive-jobs/m-p/125809#M88553</link>
    <description>&lt;P&gt;&lt;A rel="user" href="https://community.cloudera.com/users/10163/clukasik.html" nodeid="10163"&gt;@clukasik&lt;/A&gt;, &lt;A rel="user" href="https://community.cloudera.com/users/2348/divakarreddya.html" nodeid="2348"&gt;@Divakar Annapureddy&lt;/A&gt;&lt;/P&gt;&lt;P&gt;This was an issue related to the JIRA - &lt;A href="https://issues.apache.org/jira/browse/HIVE-11681"&gt;https://issues.apache.org/jira/browse/HIVE-11681&lt;/A&gt;&lt;/P&gt;&lt;P&gt;We have hive jobs that use same aux jars and "add jar" statements and these jobs execute concurrently. So, apparently, when one of the jobs is completed, the loader is closed while the other job is still using it and hence the KILL signal. We solved it by removing add jar statements from concurrent hive jobs and instead added the auxiliary jars in hive.aux.jars.path at the time of HS2 "start" which means that these jars would be loaded only once and will not be removed/closed/unloaded per session. Thus concurrent hive jobs will not have problem in accessing them.&lt;/P&gt;&lt;P&gt;To make the jars available to HS2 - they were copied on each machine that hosted HS2 and then through Ambari added the path to HIVE_AUX_JARS_PATH only for the hiveserver component in the hive-env template script.&lt;/P&gt;&lt;P&gt;if [ "$SERVICE" = "hiveserver2" ]; then &lt;/P&gt;&lt;P&gt;    CUSTOM_AUX_JARS_PATH=/&amp;lt;path_to_aux_lib_dir&amp;gt;&lt;/P&gt;&lt;P&gt;    if [ -d $CUSTOM_AUX_JARS_PATH ]; then         &lt;/P&gt;&lt;P&gt;        CUSTOM_AUX_JARS=`echo $CUSTOM_AUX_JARS_PATH/*.jar | sed 's/ /,/g'` &lt;/P&gt;&lt;P&gt;        export   HIVE_AUX_JARS_PATH="$HIVE_AUX_JARS_PATH,$CUSTOM_AUX_JARS" &lt;/P&gt;&lt;P&gt;     fi &lt;/P&gt;&lt;P&gt;fi&lt;/P&gt;</description>
    <pubDate>Sun, 08 Jan 2017 19:18:19 GMT</pubDate>
    <dc:creator>dosigary</dc:creator>
    <dc:date>2017-01-08T19:18:19Z</dc:date>
  </channel>
</rss>

