Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Tez Client Memory

Solved Go to solution

Hive Tez Client Memory

Explorer

I am trying to follow a strange problem I am seeing with Hive on Tez on the sandbox to make things simple.

When I run hive cli I see there is a container occupied in YARN via Application for the session. It however is using 500 MB of memory. I am trying to figure out where is this number coming up. The HADOOP_HEAPSIZE is set at 250MB.

The minimum container size in YARN is yarn.scheduler.minimum-allocation-mb is 250. The Tez container size in hive-site is set at 250 (hive.tez.container.size).

Even after changing hive.tez.container.size I don't see any change in the memory utilization for the single YARN Application container.

I've also tried to play around with the tez.am.resource.memory.mb and tez.dag.am.resource.memory.mb but nothing worked.

Am I missing something? Is there some kind of calculation with the HADOOP_HEAPSIZE which should be affecting the number to change?

I am trying to see if the idle session Hive cli connection memory can be brought down since they keep the resources hooked up on a small cluster.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hive Tez Client Memory

@Gagan Brahmi modify tez.am.resource.memory.mb in ambari tez configuration, restart tez and hive client.

View solution in original post

2 REPLIES 2
Highlighted

Re: Hive Tez Client Memory

@Gagan Brahmi modify tez.am.resource.memory.mb in ambari tez configuration, restart tez and hive client.

View solution in original post

Highlighted

Re: Hive Tez Client Memory

Explorer

Strangely when I tried to modify the tez.am.resource.memory.mb under Tez yesterday it wasn't taking effect. However, it does seem to be taking effect now. Appreciate your inputs.

Don't have an account?
Coming from Hortonworks? Activate your account here