Member since
05-08-2020
5
Posts
0
Kudos Received
0
Solutions
12-11-2020
05:50 AM
The titled SocketTimeoutException occurs when the thrift-client in hiveConnection object is in the process of actively reading sql results from hive server2 ( thrift server), and is not able to receive anything until the TSocket's time out occurs. You can check the source code from:HiveConnection.setupLoginTimeout and HiveAuthFactory.getSocketTransport. So you need to either tuning hiveserver2, or increase the TSocket's timeout setting. And for now, the only way to increase Tsocket's time out setting is via: DriverManager.setLoginTimeout() you can check below jira for more information: https://issues.apache.org/jira/browse/HIVE-22196 https://issues.apache.org/jira/browse/HIVE-6679 https://issues.apache.org/jira/browse/HIVE-12371
... View more
11-23-2020
02:06 AM
nope. I didn't use any custom/auxiliary jars. I am not very sure about how jars are loaded when using spark execution engine for hive, but i do notice that class path are tailored by /opt/cloudera/parcels/CDH/lib/hive/bin/hive, as below shows, to add spark related jars, while this have nothing to do with antlr-runtime-xx.jar or antlr4-runtime-xx.jar: (so i am confused why this happens for hive on spark while not for hive on mr?) # add Spark jars to the classpath if [[ -n "$SPARK_HOME" ]] then CLASSPATH=${CLASSPATH}:${SPARK_HOME}/jars/spark-core*.jar CLASSPATH=${CLASSPATH}:${SPARK_HOME}/jars/spark-unsafe*.jar CLASSPATH=${CLASSPATH}:${SPARK_HOME}/jars/scala-library*.jar fi
... View more
11-22-2020
05:55 PM
just a add on: the underneath class that caused the problem, which is org/antlr/runtime/tree/CommonTree, can't be found in antlr4-runtime-xxx.jar, but can be found in antlr-runtime-xxx.jar, as below screen shows: So we copied antlr-runtime-xxx.jar from the standard lib of hive into the standard jar lib of spark, our issue seems to be resolved by this.
... View more
11-16-2020
11:31 PM
In CDH6.X, when running hive sql using spark execution engine, sometimes i will encounter below error, while this doesn't happen in CDH5.X:
scheduler.TasksetManager: Lost task 0.1 in stage 22.0(TID 37, node03, executor 1): UnknownReason util.Utils: uncaught exception in thread task-result-getter-1 java.lang.NoClassDefFoundError: org/antlr/runtime/tree/CommonTree at java.lang.ClassLoader.defineClass1(Native.Method)
If i switch to the MR execution engine, the above error is gone.
This seems to be related to the loading of classes in antlr-runtime-xxx.jar and antlr4-runtime-xx.jar under /opt/cloudera/parcels/CDH/lib/hive/lib.
... View more
05-08-2020
10:36 PM
I am using CM 5.14.2, and from hive's configuration page, i can see the value for javax.jdo.option.ConnectionUserName, which is used to access the underneath metastore db like mysql, but i can't see the value of javax.jdo.option.ConnectionPassword (which is display as **** in the page).
i have forgotten the password used to access hive metastore db. Is there any way to find it back?
... View more
Labels: