Member since
09-16-2021
360
Posts
53
Kudos Received
28
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
121 | 02-04-2025 05:55 AM | |
527 | 11-22-2024 05:29 AM | |
261 | 11-15-2024 06:38 AM | |
544 | 11-13-2024 07:12 AM | |
576 | 11-10-2024 11:19 PM |
11-20-2024
11:29 PM
1 Kudo
@yangdkny, Did the response help resolve your query? If it did, kindly mark the relevant reply as the solution, as it will aid others in locating the answer more easily in the future.
... View more
11-17-2024
09:22 PM
@Bhavs, Did the response help resolve your query? If it did, kindly mark the relevant reply as the solution, as it will aid others in locating the answer more easily in the future.
... View more
11-15-2024
06:38 AM
1 Kudo
The below error usually occurs when there is a mismatch between the versions of Tez and Hive. Validate the compatibility. Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 112
Serialization trace:
conf (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:159)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:758)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClass(SerializationUtilities.java:188)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:117)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:129)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:877)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:183)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:235)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:42)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:796)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:221)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:124)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:129)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:774)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:213)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializeObjectByKryo(SerializationUtilities.java:838)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializePlan(SerializationUtilities.java:745)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:489)
... 19 more Additionally, it is important to validate if any auxiliary jars are being used and ensure that these jars do not conflict with the jars in the Hive classpath. if two versions of same jars are present in the classapth , which also creates a problem in the serialization.
... View more
11-13-2024
11:09 PM
1 Kudo
Cloudera Manager offers configurations and tools to optimize the usage of Tez as an execution engine for big data workloads, particularly when using Hive. The Tez CM-UI in Cloudera Manager allows for fine-tuning of Tez configurations to meet specific workload requirements, including adjusting Tez container sizes, memory settings, and parallelism levels. These Tez configurations can be applied cluster-wide, specifically for Hive. Please verify that the HIVE and Hive_ON_TEZ services are active and functioning properly. Similarly, the Core configuration applies to the entire cluster and provides an interface to change configurations cluster-wide. Ensure that all services are running smoothly. If any particular service appears as red, investigate the individual service logs for further troubleshooting.
... View more
11-13-2024
01:40 PM
@alex88 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
11-12-2024
05:27 AM
For partitionColumn, can you use Date column and date range for lower and upper bound for spark 2.4.8 ?
... View more
11-11-2024
01:11 AM
1 Kudo
Thanks for the response. I tried using these steps, I am getting the following response when I use command: Error: kafka-topics --bootstrap-server <broker:port> --list --command-config client.properties 24/11/11 08:08:21 INFO authenticator.AbstractLogin: [main]: Successfully logged in. 24/11/11 08:08:21 INFO kerberos.KerberosLogin: [kafka-kerberos-refresh-thread-null]: [Principal=null]: TGT refresh thread started. 24/11/11 08:08:21 INFO kerberos.KerberosLogin: [kafka-kerberos-refresh-thread-null]: [Principal=null]: TGT valid starting at: 2024-11-11T07:56:28.000+0000 24/11/11 08:08:21 INFO kerberos.KerberosLogin: [kafka-kerberos-refresh-thread-null]: [Principal=null]: TGT expires: 2024-11-11T17:56:28.000+0000 24/11/11 08:08:21 INFO kerberos.KerberosLogin: [kafka-kerberos-refresh-thread-null]: [Principal=null]: TGT refresh sleeping until: 2024-11-11T16:22:10.118+0000 24/11/11 08:08:21 WARN admin.AdminClientConfig: [main]: These configurations '[ssl.truststore.password.generator]' were supplied but are not used yet. 24/11/11 08:08:21 INFO utils.AppInfoParser: [main]: Kafka version: 3.4.1.7.1.9.0-387 24/11/11 08:08:21 INFO utils.AppInfoParser: [main]: Kafka commitId: b0573bcfb543760f 24/11/11 08:08:21 INFO utils.AppInfoParser: [main]: Kafka startTimeMs: 1731312501450 24/11/11 08:08:22 INFO utils.AppInfoParser: [kafka-admin-client-thread | adminclient-1]: App info kafka.admin.client for adminclient-1 unregistered 24/11/11 08:08:22 WARN kerberos.KerberosLogin: [kafka-kerberos-refresh-thread-null]: [Principal=null]: TGT renewal thread has been interrupted and will exit. 24/11/11 08:08:22 INFO metrics.Metrics: [kafka-admin-client-thread | adminclient-1]: Metrics scheduler closed 24/11/11 08:08:22 INFO metrics.Metrics: [kafka-admin-client-thread | adminclient-1]: Closing reporter org.apache.kafka.common.metrics.JmxReporter 24/11/11 08:08:22 INFO metrics.Metrics: [kafka-admin-client-thread | adminclient-1]: Metrics reporters closed
... View more
11-10-2024
11:22 PM
1 Kudo
Thanks yes it was version mismatch issue upgrading hive - 4.0 and tez 0.10.3 works forme
... View more
10-25-2024
05:10 AM
1 Kudo
From the below , it does looks like tez session itself not initialized. Validate configuration from set -v , make sure everything fine and try re-running the query. org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1726980746968_0077 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1726980746968_0077_000001 exited with exitCode: -1000
Failing this attempt.Diagnostics: [2024-10-13 10:30:27.706]Application application_1726980746968_0077 initialization failed (exitCode=255) with output: main : command provided 0 If you can't able to identify the incorrect configuration , raise a support case for the same.
... View more
10-25-2024
05:02 AM
1 Kudo
Error looks similar to HIVE-27778. Can you try the workaround given in the kb
... View more