Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Maximum Hive Tez Container Size Does Not Reflect YARN Maximum Container Size

Highlighted

Maximum Hive Tez Container Size Does Not Reflect YARN Maximum Container Size

Expert Contributor

Hi,

On my HDP 2.3.4 cluster, I have configured maximum YARN container size to 102GB and restarted related services. But on the Hive service config page on Ambari, it only allows me to configure hive.tez.container.size up to around 6GB.

Is there any limits on max Hive Tez container size?

8508-img-0451.jpg

3 REPLIES 3
Highlighted

Re: Maximum Hive Tez Container Size Does Not Reflect YARN Maximum Container Size

HI Yjiang,

I dont think the scale you see is any kind of upper limit to "Tez Container Size", you can always override it with input box(click on pencil icon) and set the value manually. But take care and keep it similar to Actual container size.

Highlighted

Re: Maximum Hive Tez Container Size Does Not Reflect YARN Maximum Container Size

Explorer

Sounds that, Tez container size cannot be increased beyond 6820 from the Scale in Ambari or by editing with the pencil.

Yes, as Sunile mentioned the Tez container Value set in the Ambari can overriden in the hive prompt, but the value needs to be mentioned as INT. (MB cannot be included).

8540-tez-value.png

output of hive> set -v;

8561-tez-value-2.png

Highlighted

Re: Maximum Hive Tez Container Size Does Not Reflect YARN Maximum Container Size

Super Guru

You can always override this on a per job or query basis using

SET hive.tez.container.size=10000MB
Don't have an account?
Coming from Hortonworks? Activate your account here