Support Questions

Find answers, ask questions, and share your expertise

Can't insert simple values into Hive table

Explorer

Hi,

when I issue the following insert command 'insert into table test values(1,"name")', I am getting the error:

jdbc:hive2://host.domain.local> insert into table test values(1,"name");
INFO  : Compiling command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30): insert into table test values(1,"name")
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col1, type:int, comment:null), FieldSchema(name:col2, type:string, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30); Time taken: 0.245 seconds
INFO  : Executing command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30): insert into table test values(1,"name")
INFO  : Query ID = hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30
INFO  : Total jobs = 1
INFO  : Launching Job 1 out of 1
INFO  : Starting task [Stage-1:MAPRED] in serial mode
INFO  : Subscribed to counters: [] for queryId: hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30
INFO  : Tez session hasn't been created yet. Opening session
INFO  : Dag name: insert into table test values(1,"name") (Stage-1)
INFO  : Status: Running (Executing on YARN cluster with App id application_1541680241474_0006)
----------------------------------------------------------------------------------------------
        VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1            container       RUNNING      1          0        0        1       4       0
Reducer 2        container        INITED      1          0        0        1       0       0
----------------------------------------------------------------------------------------------
VERTICES: 00/02  [>>--------------------------] 0%    ELAPSED TIME: 7.50 s
----------------------------------------------------------------------------------------------
ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1541680241474_0006_1_00, diagnostics=[Task failed, taskId=task_1541680241474_0006_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_0:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_1:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_2:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_3:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1541680241474_0006_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]
ERROR : Vertex killed, vertexName=Reducer 2, vertexId=vertex_1541680241474_0006_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1541680241474_0006_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]
ERROR : DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
INFO  : org.apache.tez.common.counters.DAGCounter:
INFO  :    NUM_FAILED_TASKS: 4
INFO  :    TOTAL_LAUNCHED_TASKS: 4
INFO  :    AM_CPU_MILLISECONDS: 2460
INFO  :    AM_GC_TIME_MILLIS: 0
ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1541680241474_0006_1_00, diagnostics=[Task failed, taskId=task_1541680241474_0006_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_0:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_1:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_2:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_3:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...       
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1541680241474_0006_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1541680241474_0006_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1541680241474_0006_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
INFO  : Completed executing command(queryId=hive_20181108185449_e3ac40a8-b413-442e-8cd3-3a5aaeb0da30); Time taken: 12.202 seconds
Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1541680241474_0006_1_00, diagnostics=[Task failed, taskId=task_1541680241474_0006_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_0:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_1:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        ...
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_2:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
...
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1541680241474_0006_1_00_000000_3:java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
        at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
        ...
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ...
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1541680241474_0006_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1541680241474_0006_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1541680241474_0006_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2)
0: jdbc:hive2://host.domain.local>

I have tried changing the container size in YARN, Hive, Tez but no success. I also set the recommendation values provided by yarn-utils.py but it also is not helped.

Please Help!

2 REPLIES 2

@Zholaman Kubaliyev

The issue could be related to incorrect configuration / memory value for property "tez.runtime.io.sort.mb".
Refer HCC link for tuning tez configurations.

Explorer

Hello Sindhu, you may be right. I have resolved this problem by tuning YARN, MapReduce, Tez, Hive. I can't tell what exact options impacted on the finish result but now Hive is working fine.

Thank you very much for your answer!