java lang out of memory error:

[#1]

java.pngi am trying to execute a tez job on hive and it is starting the task but is continuously failing saying java lang runtime exception: java out of memory exception.

also vertex of the dag failed due to the own task failure.

i am trying a very simple task using tez and also tried to increase the container size still the problem persists.

[#2]

@pk reddy

Can you share the complete stack of the error and also, the query being run?

[#3]

select sum(salary) from emp;

this is the query being run and on a table consisting of only 5 lines of data.

java1.pngjava2.png

[#4]

@pk reddy

can you share values for hive.tez.container.size and hive.tez.java.opts, tez.runtime.io.sort.mb, tez.runtime.unordered.output.buffer.size-mb.

[#5]

tez runtime.io.sort.mb=150mb

tez.runtime.unordered.output.buffer.size-mb = 844mb

hive.tez.container.size=2000mb

hive.tez.java.opts-1600mb

[#6]

@rtrivedi

[#7]

@pk reddy

Can you try with below settings?

set hive.tez.container.size=2048;

set tez runtime.io.sort.mb=512;

set tez.runtime.unordered.output.buffer.size-mb=800;

set hive.tez.java.opts=-Xmx1644m;

[#8]

i tried with the parameters that you have mentioned, still the problem persists

[#9]

@pk reddy

Can you send the below details from your cluster:

"yarn.nodemanager.resource.memory-mb"

# free -m

By any chance are you using the sandbox ?