Insert into table test values('a', 'b'); not working on Hive with tez execution engine

rsurti — Mon, 11 Nov 2024 07:15:18 GMT

Hi,

I am trying to run simple query on Hive with execution engine as Tez,

Here is the query

Insert into table test values('a', 'b');

there are only two columns in table and both are string columns.

If i insert into int column then it works fine.

also if i insert through load data command, this also works fine with string command.

Below is the error that i am getting in yarn logs:

ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1731065811939_0001_1_00, diagnostics=[Vertex vertex_1731065811939_0001_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed, vertex=vertex_1731065811939_0001_1_00 [Map 1], java.lang.RuntimeException: Failed to load plan: hdfs://localhost:9000/tmp/hive/hadoopuser/a86dedab-ce0f-4f06-8056-c37aef3cca55/hive_2024-11-11_12-26-32_369_2945486912443049901-2/hadoopuser/_tez_scratch_dir/8d991848-0c76-4ffa-a82e-76f0ca5140e5/map.xml
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:525)
at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:369)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:480)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:472)
at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:321)
at org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:121)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 112
Serialization trace:
conf (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:159)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:758)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClass(SerializationUtilities.java:188)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:117)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:129)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:877)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:183)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:235)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:42)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:796)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:221)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:124)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:129)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:774)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:213)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializeObjectByKryo(SerializationUtilities.java:838)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializePlan(SerializationUtilities.java:745)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:489)
... 19 more
]

Can anyone guide here?

Re: Insert into table test values('a', 'b'); not working on Hive with tez execution engine

ggangadharan — Fri, 15 Nov 2024 14:38:45 GMT

The below error usually occurs when there is a mismatch between the versions of Tez and Hive. Validate the compatibility.

Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 112 Serialization trace: conf (org.apache.hadoop.hive.ql.exec.TableScanOperator) aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:159) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:758) at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClass(SerializationUtilities.java:188) at org.apache.hive.com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:117) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:129) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:877) at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:183) at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:235) at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:42) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:796) at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:221) at org.apache.hive.com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:124) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:129) at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:774) at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:213) at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializeObjectByKryo(SerializationUtilities.java:838) at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializePlan(SerializationUtilities.java:745) at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:489) ... 19 more

Additionally, it is important to validate if any auxiliary jars are being used and ensure that these jars do not conflict with the jars in the Hive classpath. if two versions of same jars are present in the classapth , which also creates a problem in the serialization.

question Re: Insert into table test values('a', 'b'); not working on Hive with tez execution engine in Support Questions

Insert into table test values('a', 'b'); not working on Hive with tez execution engine

Re: Insert into table test values('a', 'b'); not working on Hive with tez execution engine