question Re: unable to connect thrift server using beeline in Support Questions

unable to connect thrift server using beeline

soumya — Fri, 18 Oct 2019 10:47:31 GMT

when am connecting to thrift server using beeline it's showing below error

Beeline version 1.2.1.spark2 by Apache Hive
beeline> !connect jdbc:hive2://host:10000
Connecting to jdbc:hive2://host:10000
Enter username for jdbc:hive2://host:10000: admin-sbarik2
Enter password for jdbc:hive2://host:10000: ********
org/apache/hive/service/cli/thrift/TCLIService$Iface
0: jdbc:hive2://host:10000 (closed)>

it's automatically closed.

Re: unable to connect thrift server using beeline

Shelton — Fri, 18 Oct 2019 20:04:10 GMT

@soumya

Have you tried the below method?

$ beeline -u jdbc:hive2://osaka.com:10000 -n hive -p hive
.........
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://osaka.com:10000
Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
0: jdbc:hive2://osaka.com:10000> show databases;
INFO : Compiling command(queryId=hive_20191018215530_3f94b050-d36c-46c9-9582-40a0fef9b6e2): show databases
...........
INFO : Completed executing command(queryId=hive_20191018215530_3f94b050-d36c-46c9-9582-40a0fef9b6e2); Time taken: 0.037 seconds
INFO : OK
+---------------------+
| database_name |
+---------------------+
| default |
| information_schema |
| sparktest |
| sys |
+---------------------+
4 rows selected (0.392 seconds)
0: jdbc:hive2://osaka.com:10000>

Re: unable to connect thrift server using beeline

soumya — Sat, 19 Oct 2019 05:30:38 GMT

no luck same issue it's automatically closed

Connecting to beeline
-n
log4j:WARN No appenders could be found for logger (org.apache.hive.beeline.SQLCompleter).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
0: beeline (closed)>

Re: unable to connect thrift server using beeline

Shelton — Sat, 19 Oct 2019 08:51:28 GMT

@soumya

Following from my previous responses can you get the connection string like below

Ambari UI --> Hive --> Summary (Tab) --> "HiveServer2 JDBC URL" --> Click on the at the right side of the URL to copy the URL and then try this URL with beeline once
At the prompt add beeline to the copied link above.

# beeline jdbc:hive2://osaka.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://osaka.com:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Enter username for jdbc:hive2://osaka.com:2181/default: hive
Enter password for jdbc:hive2://osaka.com:2181/default: ****
19/10/19 09:40:50 [main]: INFO jdbc.HiveConnection: Connected to osaka.com:10000
Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
0: jdbc:hive2://osaka.com:2181/default> show tables;
INFO : Compiling command(queryId=hive_20191019094333_357978cc-2f7d-433f-badc-6d8f20178361): show tables
..........
INFO : Completed executing command(queryId=hive_20191019094333_357978cc-2f7d-433f-badc-6d8f20178361); Time taken: 11.262 seconds
INFO : OK
+-------------+
| tab_name |
+-------------+
| hello_acid |
+-------------+
1 row selected (77.143 seconds)
0: jdbc:hive2://osaka.com:2181/default>

Specifying a database in the connect string

# beeline -u "jdbc:hive2://osaka.com:2181/sparktest;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"
SLF4J: Class path contains multiple SLF4J bindings.
...........
Connecting to jdbc:hive2://osaka.com:2181/sparktest;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
19/10/19 10:01:21 [main]: INFO jdbc.HiveConnection: Connected to osaka.com:10000
Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
0: jdbc:hive2://osaka.com:2181/sparktest> show tables;

Methods of Hs2 connections

HiveServer2 and a JDBC client (such as Beeline) as the primary way to access Hive. It uses SQL standard-based authorization or Ranger-based authorization. However if wish to access Hive data from other applications, such as Pig. For these use cases, use the Hive CLI and storage-based authorization.
HS2 using Binary Transport Mode

beeline jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2HiveServer2

HS2 using HTTP Transport Mode

beeline jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;transportMode=http;httpPath=cliservice

HS2 Interactive note zooKeeperNamespace=hiveserver2-hive2 in the below URL

beeline jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-hive2

HS2 Scripts using Beeline, you can make use of -f option in Beeline.

beeline -u "jdbc:hive2://master01:2181,master02:2181,master03:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" -f file.hql

The above methods should succeed.

Re: unable to connect thrift server using beeline

soumya — Sat, 19 Oct 2019 12:52:08 GMT

can you share before i connecting to spark thrift server with beeline what configuration need to check ...???

what i think something wrong with configuration file....

Re: unable to connect thrift server using beeline

soumya — Sat, 19 Oct 2019 13:01:57 GMT

facing below error

#-> ./bin/beeline
Beeline version 1.2.1.spark2 by Apache Hive
beeline> !connect jdbc:hive2://host:10001
Connecting to jdbc:hive2://host:10001
Enter username for jdbc:hive2://host:10001: SBarik2
Enter password for jdbc:hive2://host:10001: ********
org/apache/hive/service/cli/thrift/TCLIService$Iface
0: jdbc:hive2://host:10001 (closed)>

can you share what configuration need to check

Re: unable to connect thrift server using beeline

Shelton — Sat, 19 Oct 2019 18:22:21 GMT

@soumya

Can you share the output od the below

Ambari UI --> Hive --> Summary (Tab) --> "HiveServer2 JDBC URL" --> Click on the at the right side of the URL to copy the URL

I see you using port 10001 ???

What is the output from your Linux console of

$ hostname -f

You should append that derived value i.e soumya.com to the connect string

Re: unable to connect thrift server using beeline

soumya — Sun, 20 Oct 2019 08:53:45 GMT

please suggest facing below issue

Beeline version 1.2.1.spark2 by Apache Hive
beeline> !connect jdbc:hive2//awsdev2185:10000
scan complete in 0ms
19/10/20 04:49:36 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: minlog-1.2.jar (No such file or directory)
19/10/20 04:49:37 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: objenesis-1.2.jar (No such file or directory)
19/10/20 04:49:37 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: reflectasm-1.07-shaded.jar (No such file or directory)
scan complete in 1177ms
No known driver to handle "jdbc:hive2//awsdev2185:10000"

Re: unable to connect thrift server using beeline

Shelton — Sun, 20 Oct 2019 09:22:39 GMT

@soumya

when I ask you to share output there is a valid reason for that but when you continue to send me errors I can figure out what could be the issue because the errors are different each time.

In all my previous posting I requested outputs or screenshots !!

Like the Hs2 connect string from the Ambari UI, /etc/hosts and output of $ hostname -f, etc

What is the backend database you are using for hive?

Re: unable to connect thrift server using beeline

soumya — Sun, 20 Oct 2019 13:34:18 GMT

backend database am using spark-sql

Re: unable to connect thrift server using beeline

Shelton — Sun, 20 Oct 2019 13:53:52 GMT

@soumya

Spark SQL is Spark’s interface for working with structured and semi-structured data so it is not what I asked Hive Metastore uses Mysql; MariaDB, SQL Server, Oracle etc for its metastore !!
If you can't provide infor requested it will be difficult to help you out!

Re: unable to connect thrift server using beeline

soumya — Mon, 21 Oct 2019 07:50:23 GMT

when am giving below connect ion string showing below error

beeline> !connect jdbc:hive2//awshv2200:10000
scan complete in 0ms
19/10/21 03:40:04 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: minlog-1.2.jar (No such file or directory)
19/10/21 03:40:04 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: objenesis-1.2.jar (No such file or directory)
19/10/21 03:40:04 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: reflectasm-1.07-shaded.jar (No such file or directory)
scan complete in 553ms
No known driver to handle "jdbc:hive2//awshv2200:10000"

please note as my configuration in hive-site.xml the transport mode is binary

can you share me the the connection string when transport mode is binary

am using same same host i;e

am currently login with host awshv2200 and my spark thrift server host is awshv2200 and port no-10000

Re: unable to connect thrift server using beeline

soumya — Mon, 21 Oct 2019 08:05:38 GMT

please find the configuration

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>oracle.jdbc.OracleDriver</value>
</property>

<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>

please share your suggestion what need to check and configuration

Re: unable to connect thrift server using beeline

Shelton — Mon, 21 Oct 2019 19:49:19 GMT

@soumya

From your property files, your backend database is Oracle! There is something very weird about the information you are sharing with members that's why we can't resolve your problem?

Can you get the value of your hive-site.xml "javax.jdo.option.ConnectionURL" that should confirm oracle

All along you claimed you were using MySQL database for your metastore? But the connect string to an oracle database is different,here is the syntax for connecting

"jdbc:oracle:thin:@localhost/remote_host:port:db_name" so according to your shared hive-site.xml correct connect is

"jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033"

"jdbc:oracle:thin:@<hoostname[DFJHNNJHJUUI]>:port[4355]:db_name[KKJH0033]

Else connecting Direct - Binary Transport Mode example below
beeline -n hive -p hive -u "jdbc:hive2://osaka.com:10000/sparktest"

Ranger_using_Mysql_db

Hs2 mode ports

Connecting to my Sparktest database

Hive_using_Mysql_metastore

Please clarify

Re: unable to connect thrift server using beeline

soumya — Tue, 22 Oct 2019 07:41:45 GMT

Thanks for your support but no luck facing same issue connection closed automatically.

can you help me for an important shell script

write a shell script to connect spark thrift server using beeline...

request you please share the syntax .....

Re: unable to connect thrift server using beeline

soumya — Tue, 22 Oct 2019 08:06:08 GMT

As per you previous conversation

All along you claimed you were using MySQL database for your metastore? But the connect string to an oracle database is different,here is the syntax for connecting

"jdbc:oracle:thin:@localhost/remote_host:port:db_name" so according to your shared hive-site.xml correct connect is

"jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033"

"jdbc:oracle:thin:@<hoostname[DFJHNNJHJUUI]>:port[4355]:db_name[KKJH0033]

Else connecting Direct - Binary Transport Mode example below
beeline -n hive -p hive -u "jdbc:hive2://osaka.com:10000/sparktest"

-----------------------------------------------------------------------------------------------------------------------------

as per my current configuration file

<property>
<name>hive.server2.authentication</name>
<value>LDAP</value>
</property>
<property>
<name>spark.sql.hive.thriftServer.adf.authenticationProvider</name>
<value>com.dmz.adf.analyze.spark2adf.thriftserver.auth.hnzLdapAuthenticationProvider</value>
</property>

2. am using 10000 port

3. connection url is "jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033" and also i have user id and password

4. Transport mode is binary

5. my thrift srver host is (awsdev3344 )

am using awsdev3344 server for connection to thrift server (awsdev3344 ) using beeline

am using below connection string

Beeline version 1.2.1.spark2 by Apache Hive
beeline>!connect jdbc:hive2://awsdev3344 :10000

but connection is closed automatically.

i already share you the connection details please share the connection string transport mode is binary so that i can process

Re: unable to connect thrift server using beeline

rohitmalhotra — Wed, 23 Oct 2019 12:01:33 GMT

Hi @soumya,

It seems there is some confusion here.

As per my understanding, you are trying to connect to Spark Thrift Server via beeline. Please correct me, if I am wrong.

You you to be able to connect to Spark Thrift server via beeline, you need to make sure you are providing the correct hostname and portnumber in the JDBC URL you are using in the beeline.

For example:

jdbc:hive2://host:portnumber/

Here, the "host" will be the Hostname of the server where Spark Thriftserver is running.

Let us say it is running on abc.soumya.com.

The default portnumber for Spark Thriftserver is 10000.

But this portnumber can be configured to something else as well. You need to find the correct portnumber.

Thus, you connect string would look like below:

jdbc:hive2://abc.soumya.com:10000/

You can refer the below link for more information on this:

https://spark.apache.org/docs/latest/sql-distributed-sql-engine.html

Re: unable to connect thrift server using beeline

soumya — Wed, 23 Oct 2019 14:02:20 GMT

THANKS FOR THE INFO.

ISSUE RESOLVE .

Re: unable to connect thrift server using beeline

Shelton — Wed, 23 Oct 2019 15:27:20 GMT

@soumya

Good to hear that can you share what solution worked for you this way other who encounter the same problem can quickly resolve it.

Thats what we csll community contribution

Happy hadooping

Re: unable to connect thrift server using beeline

rohitmalhotra — Thu, 24 Oct 2019 06:34:51 GMT

Hi @soumya,

Can you please confirm exactly how the issue was resolved.

Please do accept the answer which helped you.