Support Questions
Find answers, ask questions, and share your expertise

java lang out of memory error:

Explorer

java.pngi am trying to execute a tez job on hive and it is starting the task but is continuously failing saying java lang runtime exception: java out of memory exception.

also vertex of the dag failed due to the own task failure.

i am trying a very simple task using tez and also tried to increase the container size still the problem persists.

8 REPLIES 8

@pk reddy

Can you share the complete stack of the error and also, the query being run?

Explorer

select sum(salary) from emp;

this is the query being run and on a table consisting of only 5 lines of data.

java1.pngjava2.png

Contributor

@pk reddy

can you share values for hive.tez.container.size and hive.tez.java.opts, tez.runtime.io.sort.mb, tez.runtime.unordered.output.buffer.size-mb.

Explorer

tez runtime.io.sort.mb=150mb

tez.runtime.unordered.output.buffer.size-mb = 844mb

hive.tez.container.size=2000mb

hive.tez.java.opts-1600mb

Explorer

@pk reddy

Can you try with below settings?

set hive.tez.container.size=2048;

set tez runtime.io.sort.mb=512;

set tez.runtime.unordered.output.buffer.size-mb=800;

set hive.tez.java.opts=-Xmx1644m;

Explorer

i tried with the parameters that you have mentioned, still the problem persists

Rising Star

@pk reddy

Can you send the below details from your cluster:

"yarn.nodemanager.resource.memory-mb"

# free -m

By any chance are you using the sandbox ?

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.