Created 11-08-2018 01:18 PM
Hi,
when I issue the following insert command 'insert into table test values(1,"name")', I am getting the error:
jdbc:hive2://host.domain.local> insert into table test values(1,"name"); INFO : Compiling command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30): insert into table test values(1,"name") INFO : Semantic Analysis Completed (retrial = false) INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col1, type:int, comment:null), FieldSchema(name:col2, type:string, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30); Time taken: 0.245 seconds INFO : Executing command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30): insert into table test values(1,"name") INFO : Query ID = hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30 INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode INFO : Subscribed to counters: [] for queryId: hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30 INFO : Tez session hasn't been created yet. Opening session INFO : Dag name: insert into table test values(1,"name") (Stage-1) INFO : Status: Running (Executing on YARN cluster with App id application_1541680241474_0006) ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 container RUNNING 1 0 0 1 4 0 Reducer 2 container INITED 1 0 0 1 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 7.50 s ---------------------------------------------------------------------------------------------- ERROR : Status: Failed ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1541680241474_0006_1_00, diagnostics=[Task failed, taskId=task_1541680241474_0006_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_0:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_1:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_2:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_3:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1541680241474_0006_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE] ERROR : Vertex killed, vertexName=Reducer 2, vertexId=vertex_1541680241474_0006_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1541680241474_0006_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE] ERROR : DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 INFO : org.apache.tez.common.counters.DAGCounter: INFO : NUM_FAILED_TASKS: 4 INFO : TOTAL_LAUNCHED_TASKS: 4 INFO : AM_CPU_MILLISECONDS: 2460 INFO : AM_GC_TIME_MILLIS: 0 ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1541680241474_0006_1_00, diagnostics=[Task failed, taskId=task_1541680241474_0006_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_0:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_1:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_2:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_3:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1541680241474_0006_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1541680241474_0006_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1541680241474_0006_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 INFO : Completed executing command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30); Time taken: 12.202 seconds Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1541680241474_0006_1_00, diagnostics=[Task failed, taskId=task_1541680241474_0006_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_0:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_1:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) ... at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_2:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_3:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177) ... at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1541680241474_0006_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1541680241474_0006_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1541680241474_0006_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2) 0: jdbc:hive2://host.domain.local>
I have tried changing the container size in YARN, Hive, Tez but no success. I also set the recommendation values provided by yarn-utils.py but it also is not helped.
Please Help!
Created 11-12-2018 08:26 AM
The issue could be related to incorrect configuration / memory value for property "tez.runtime.io.sort.mb".
Refer HCC link for tuning tez configurations.
Created 11-12-2018 09:56 AM
Hello Sindhu, you may be right. I have resolved this problem by tuning YARN, MapReduce, Tez, Hive. I can't tell what exact options impacted on the finish result but now Hive is working fine.
Thank you very much for your answer!