Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive LLap TezTask Error task memory while running task

Hive LLap TezTask Error task memory while running task

New Contributor

Hi you,

I'm facing with these error:

Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1477624725642_0102_1_00, diagnostics=[Task failed, taskId=task_1477624725642_0102_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1477624725642_0102_1_00_000000_0:java.lang.IllegalArgumentException: tez.runtime.io.sort.mb 300 should be larger than 0 and should be less than the available task memory (MB):174

If I decrease tez.runtime.io.sort.mb < 300 , It's ok.

But how I can increase "task memory" ?

Thanks in advance,

3 REPLIES 3

Re: Hive LLap TezTask Error task memory while running task

Rising Star

Any solution for this problem?

Re: Hive LLap TezTask Error task memory while running task

New Contributor

Hi You,

You should set LLAP heap size larger for increasing the available task memory.

Highlighted

Re: Hive LLap TezTask Error task memory while running task

Contributor

I had same issue

attempt_1525249294393_0004_1_02_000000_3:java.lang.IllegalArgumentException: tez.runtime.io.sort.mb 819 should be larger than 0 and should be less than the available task memory (MB):486 at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) at org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.getInitialMemoryRequirement(ExternalSorter.java:351) at org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.initialize(OrderedPartitionedKVOutput.java:93) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable._callInternal(LogicalIOProcessorRuntimeTask.java:533) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:516) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:501) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

"

This is resolved after increasing LLAP daemon heap size.

Thanks @Tran Quyet Thang for the solution.