Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive job fail on TEZ due to out of memory..

avatar
Expert Contributor

Hi one of my job in hive on Tez gets fail at 98% due tojava.lang.OutOfMemoryError:Java heap space,

i am using HDP 2.1.2

i have below values

mapreduce.map.memory.mb=3584;

mapreduce.map.java.opts=-Xmx2560m;

mapreduce.reduce.memory.mb=3584;

mapreduce.reduce.java.opts=-Xmx2560m

Can any one help me to out this error is great appreciated..

Regards

sankar

1 ACCEPTED SOLUTION

avatar
Contributor

Your settings are mapreduce settings which may be not recognized by tez. Try setting tez config 'tez.task.resource.memory.mb' and 'tez.task.launch.cmd-opts' corresppondingly.

View solution in original post

5 REPLIES 5

avatar
Guru

@rama

did you try increasing your mappers memory? and is your request failing with hive.execution.engine=mr as well?

avatar
Contributor

Your settings are mapreduce settings which may be not recognized by tez. Try setting tez config 'tez.task.resource.memory.mb' and 'tez.task.launch.cmd-opts' corresppondingly.

avatar
Explorer

I had similar issues on the Sandbox 2.5, increased

hive.tez.container.size to 1024 and

hive.tez.java.opts to Xmx8192m OOM issues went away.

avatar
Expert Contributor

Thank you all i have increesed all the above values to 2mb more..now job is working fine..

avatar
Master Mentor

Please select the best answer to close out the thread