Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

java lang out of memory error:

java lang out of memory error:

New Contributor

java.pngi am trying to execute a tez job on hive and it is starting the task but is continuously failing saying java lang runtime exception: java out of memory exception.

also vertex of the dag failed due to the own task failure.

i am trying a very simple task using tez and also tried to increase the container size still the problem persists.

8 REPLIES 8

Re: java lang out of memory error:

@pk reddy

Can you share the complete stack of the error and also, the query being run?

Re: java lang out of memory error:

New Contributor

select sum(salary) from emp;

this is the query being run and on a table consisting of only 5 lines of data.

java1.pngjava2.png

Re: java lang out of memory error:

Contributor

@pk reddy

can you share values for hive.tez.container.size and hive.tez.java.opts, tez.runtime.io.sort.mb, tez.runtime.unordered.output.buffer.size-mb.

Re: java lang out of memory error:

New Contributor

tez runtime.io.sort.mb=150mb

tez.runtime.unordered.output.buffer.size-mb = 844mb

hive.tez.container.size=2000mb

hive.tez.java.opts-1600mb

Re: java lang out of memory error:

New Contributor

Re: java lang out of memory error:

@pk reddy

Can you try with below settings?

set hive.tez.container.size=2048;

set tez runtime.io.sort.mb=512;

set tez.runtime.unordered.output.buffer.size-mb=800;

set hive.tez.java.opts=-Xmx1644m;

Re: java lang out of memory error:

New Contributor

i tried with the parameters that you have mentioned, still the problem persists

Highlighted

Re: java lang out of memory error:

Rising Star

@pk reddy

Can you send the below details from your cluster:

"yarn.nodemanager.resource.memory-mb"

# free -m

By any chance are you using the sandbox ?