Skip to main content
ryu
ryu
Gold ⭐️
May 27, 2021
Question

hiveserver2 error

Hi experts,

Many of the devs are getting the following error when trying to connect with squirrel:

Error: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out

SQLState:  08S01

ErrorCode: 0

 

I believe the issue may be because there are too many concurrent hive connections.

 

Can someone help suggest how should i configure this to allow more hive connections?

Also I find it weird that I can connect through beeline using the edge node of that cluster but Squirrel is giving an error.

 

Thanks,

Any help is much appreciated.

12 replies

heya
heya
Silver ⭐️
May 28, 2021

all connections from squirrel got the issues?

Translate
Shelton
Shelton
Gold ⭐️⭐️⭐️
May 28, 2021

@ryu 
Please have a look at this document it could help you

https://serverliving.wordpress.com/2016/05/05/step-by-step-connect-squirrelsql-hiveserver2/

Translate
LeetKrew93
Gold ⭐️
LeetKrew93
Gold ⭐️
May 28, 2021

Could you please monitor the number of connections at the time you are getting this exception ?

 

Based on the number of connections, you should tune HMS and HS2 => https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cdh_ig_hive_metastore_configure.html#concept_jsw_bnc_rp

 

Also, how many HS2 instances are installed on your cluster as Cloudera recommends splitting HiveServer2 into multiple instances and load balancing them once you start allocating more than 16 GB to HiveServer2. The objective is to adjust the size to reduce the impact of Java garbage collection on active processing by the service.

Translate
ryu
ryuAuthor
Gold ⭐️
May 28, 2021

@LeetKrew93  We currently have 2 instances of HMS and HS2.

I believe if the jdbc string includes all the zookeepers, doesn't the zookeeper help load balance for hive?

Also I believe for our dev cluster, there are lots of connections. Is there a way to increase the max number of connections or should i create an extra hive instance?

Translate
ryu
ryuAuthor
Gold ⭐️
May 28, 2021

So i just counted our connections and it seems like we have around 40 connections going on at the same time.

 

So when should or how do we know how many hive instances we should have?

Translate
Terms & ConditionsAccessibility statement