I am trying to follow a strange problem I am seeing with Hive on Tez on the sandbox to make things simple.
When I run hive cli I see there is a container occupied in YARN via Application for the session. It however is using 500 MB of memory. I am trying to figure out where is this number coming up. The HADOOP_HEAPSIZE is set at 250MB.
The minimum container size in YARN is yarn.scheduler.minimum-allocation-mb is 250. The Tez container size in hive-site is set at 250 (hive.tez.container.size).
Even after changing hive.tez.container.size I don't see any change in the memory utilization for the single YARN Application container.
I've also tried to play around with the tez.am.resource.memory.mb and tez.dag.am.resource.memory.mb but nothing worked.
Am I missing something? Is there some kind of calculation with the HADOOP_HEAPSIZE which should be affecting the number to change?
I am trying to see if the idle session Hive cli connection memory can be brought down since they keep the resources hooked up on a small cluster.
