Created on
10-18-2019
03:40 AM
- last edited on
10-18-2019
03:47 AM
by
VidyaSargur
when am connecting to thrift server using beeline it's showing below error
Beeline version 1.2.1.spark2 by Apache Hive
beeline> !connect jdbc:hive2://host:10000
Connecting to jdbc:hive2://host:10000
Enter username for jdbc:hive2://host:10000: admin-sbarik2
Enter password for jdbc:hive2://host:10000: ********
org/apache/hive/service/cli/thrift/TCLIService$Iface
0: jdbc:hive2://host:10000 (closed)>
it's automatically closed.
Created 10-24-2019 12:55 AM
actually in oracle Database previously am not entry my user id and password
after entry the same connection is established. It's a minor issue
thanks for your support.
Created 10-18-2019 01:04 PM
Have you tried the below method?
$ beeline -u jdbc:hive2://osaka.com:10000 -n hive -p hive
.........
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://osaka.com:10000
Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
0: jdbc:hive2://osaka.com:10000> show databases;
INFO : Compiling command(queryId=hive_20191018215530_3f94b050-d36c-46c9-9582-40a0fef9b6e2): show databases
...........
INFO : Completed executing command(queryId=hive_20191018215530_3f94b050-d36c-46c9-9582-40a0fef9b6e2); Time taken: 0.037 seconds
INFO : OK
+---------------------+
| database_name |
+---------------------+
| default |
| information_schema |
| sparktest |
| sys |
+---------------------+
4 rows selected (0.392 seconds)
0: jdbc:hive2://osaka.com:10000>
Created 10-18-2019 10:30 PM
no luck same issue it's automatically closed
Connecting to beeline
-n
log4j:WARN No appenders could be found for logger (org.apache.hive.beeline.SQLCompleter).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
0: beeline (closed)>
Created 10-19-2019 01:51 AM
Following from my previous responses can you get the connection string like below
Ambari UI --> Hive --> Summary (Tab) --> "HiveServer2 JDBC URL" --> Click on the at the right side of the URL to copy the URL and then try this URL with beeline once
At the prompt add beeline to the copied link above.
# beeline jdbc:hive2://osaka.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://osaka.com:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Enter username for jdbc:hive2://osaka.com:2181/default: hive
Enter password for jdbc:hive2://osaka.com:2181/default: ****
19/10/19 09:40:50 [main]: INFO jdbc.HiveConnection: Connected to osaka.com:10000
Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
0: jdbc:hive2://osaka.com:2181/default> show tables;
INFO : Compiling command(queryId=hive_20191019094333_357978cc-2f7d-433f-badc-6d8f20178361): show tables
..........
INFO : Completed executing command(queryId=hive_20191019094333_357978cc-2f7d-433f-badc-6d8f20178361); Time taken: 11.262 seconds
INFO : OK
+-------------+
| tab_name |
+-------------+
| hello_acid |
+-------------+
1 row selected (77.143 seconds)
0: jdbc:hive2://osaka.com:2181/default>
Specifying a database in the connect string
# beeline -u "jdbc:hive2://osaka.com:2181/sparktest;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"
SLF4J: Class path contains multiple SLF4J bindings.
...........
Connecting to jdbc:hive2://osaka.com:2181/sparktest;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
19/10/19 10:01:21 [main]: INFO jdbc.HiveConnection: Connected to osaka.com:10000
Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
0: jdbc:hive2://osaka.com:2181/sparktest> show tables;
Methods of Hs2 connections
HiveServer2 and a JDBC client (such as Beeline) as the primary way to access Hive. It uses SQL standard-based authorization or Ranger-based authorization. However if wish to access Hive data from other applications, such as Pig. For these use cases, use the Hive CLI and storage-based authorization.
HS2 using Binary Transport Mode
beeline jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2HiveServer2
HS2 using HTTP Transport Mode
beeline jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;transportMode=http;httpPath=cliservice
HS2 Interactive note zooKeeperNamespace=hiveserver2-hive2 in the below URL
beeline jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-hive2
HS2 Scripts using Beeline, you can make use of -f option in Beeline.
beeline -u "jdbc:hive2://master01:2181,master02:2181,master03:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" -f file.hql
The above methods should succeed.
Created 10-19-2019 05:52 AM
can you share before i connecting to spark thrift server with beeline what configuration need to check ...???
what i think something wrong with configuration file....
Created on 10-19-2019 07:08 AM - edited 10-19-2019 11:22 AM
Can you share the output od the below
Ambari UI --> Hive --> Summary (Tab) --> "HiveServer2 JDBC URL" --> Click on the at the right side of the URL to copy the URL
I see you using port 10001 ???
What is the output from your Linux console of
$ hostname -f
You should append that derived value i.e soumya.com to the connect string
Created 10-19-2019 06:01 AM
facing below error
#-> ./bin/beeline
Beeline version 1.2.1.spark2 by Apache Hive
beeline> !connect jdbc:hive2://host:10001
Connecting to jdbc:hive2://host:10001
Enter username for jdbc:hive2://host:10001: SBarik2
Enter password for jdbc:hive2://host:10001: ********
org/apache/hive/service/cli/thrift/TCLIService$Iface
0: jdbc:hive2://host:10001 (closed)>
can you share what configuration need to check
Created on 10-20-2019 01:52 AM - edited 10-20-2019 01:53 AM
please suggest facing below issue
Beeline version 1.2.1.spark2 by Apache Hive
beeline> !connect jdbc:hive2//awsdev2185:10000
scan complete in 0ms
19/10/20 04:49:36 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: minlog-1.2.jar (No such file or directory)
19/10/20 04:49:37 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: objenesis-1.2.jar (No such file or directory)
19/10/20 04:49:37 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: reflectasm-1.07-shaded.jar (No such file or directory)
scan complete in 1177ms
No known driver to handle "jdbc:hive2//awsdev2185:10000"
Created 10-20-2019 02:22 AM
when I ask you to share output there is a valid reason for that but when you continue to send me errors I can figure out what could be the issue because the errors are different each time.
In all my previous posting I requested outputs or screenshots !!
Like the Hs2 connect string from the Ambari UI, /etc/hosts and output of $ hostname -f, etc
What is the backend database you are using for hive?
Created 10-20-2019 06:34 AM
backend database am using spark-sql
Created 10-20-2019 06:53 AM
Spark SQL is Spark’s interface for working with structured and semi-structured data so it is not what I asked Hive Metastore uses Mysql; MariaDB, SQL Server, Oracle etc for its metastore !!
If you can't provide infor requested it will be difficult to help you out!
Created 10-21-2019 12:50 AM
when am giving below connect ion string showing below error
beeline> !connect jdbc:hive2//awshv2200:10000
scan complete in 0ms
19/10/21 03:40:04 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: minlog-1.2.jar (No such file or directory)
19/10/21 03:40:04 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: objenesis-1.2.jar (No such file or directory)
19/10/21 03:40:04 ERROR ClassNameCompleter: Fail to parse the class name from the Jar file due to the exception:java.io.FileNotFoundException: reflectasm-1.07-shaded.jar (No such file or directory)
scan complete in 553ms
No known driver to handle "jdbc:hive2//awshv2200:10000"
please note as my configuration in hive-site.xml the transport mode is binary
can you share me the the connection string when transport mode is binary
am using same same host i;e
am currently login with host awshv2200 and my spark thrift server host is awshv2200 and port no-10000
Created 10-21-2019 01:05 AM
please find the configuration
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>oracle.jdbc.OracleDriver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
please share your suggestion what need to check and configuration
Created on 10-21-2019 12:18 PM - edited 10-21-2019 12:49 PM
From your property files, your backend database is Oracle! There is something very weird about the information you are sharing with members that's why we can't resolve your problem?
Can you get the value of your hive-site.xml "javax.jdo.option.ConnectionURL" that should confirm oracle
All along you claimed you were using MySQL database for your metastore? But the connect string to an oracle database is different,here is the syntax for connecting
"jdbc:oracle:thin:@localhost/remote_host:port:db_name" so according to your shared hive-site.xml correct connect is
"jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033"
"jdbc:oracle:thin:@<hoostname[DFJHNNJHJUUI]>:port[4355]:db_name[KKJH0033]
Else connecting Direct - Binary Transport Mode example below
beeline -n hive -p hive -u "jdbc:hive2://osaka.com:10000/sparktest"
Ranger_using_Mysql_db
Hs2 mode ports
Connecting to my Sparktest database
Hive_using_Mysql_metastore
Please clarify
Created 10-22-2019 12:41 AM
Thanks for your support but no luck facing same issue connection closed automatically.
can you help me for an important shell script
write a shell script to connect spark thrift server using beeline...
request you please share the syntax .....
Created 10-22-2019 01:06 AM
As per you previous conversation
All along you claimed you were using MySQL database for your metastore? But the connect string to an oracle database is different,here is the syntax for connecting
"jdbc:oracle:thin:@localhost/remote_host:port:db_name" so according to your shared hive-site.xml correct connect is
"jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033"
"jdbc:oracle:thin:@<hoostname[DFJHNNJHJUUI]>:port[4355]:db_name[KKJH0033]
Else connecting Direct - Binary Transport Mode example below
beeline -n hive -p hive -u "jdbc:hive2://osaka.com:10000/sparktest"
-----------------------------------------------------------------------------------------------------------------------------
as per my current configuration file
<configuration>
<!-- hnj ThriftServer authentication ldap -->
<property>
<name>hive.server2.authentication</name>
<value>LDAP</value>
</property>
<property>
<name>spark.sql.hive.thriftServer.adf.authenticationProvider</name>
<value>com.dmz.adf.analyze.spark2adf.thriftserver.auth.hnzLdapAuthenticationProvider</value>
</property>
2. am using 10000 port
3. connection url is "jdbc:oracle:thin:@DFJHNNJHJUUI:4355:KKJH0033" and also i have user id and password
4. Transport mode is binary
5. my thrift srver host is (awsdev3344 )
am using awsdev3344 server for connection to thrift server (awsdev3344 ) using beeline
am using below connection string
Beeline version 1.2.1.spark2 by Apache Hive
beeline>!connect jdbc:hive2://awsdev3344 :10000
but connection is closed automatically.
i already share you the connection details please share the connection string transport mode is binary so that i can process
Created 10-23-2019 05:01 AM
Hi @soumya,
It seems there is some confusion here.
As per my understanding, you are trying to connect to Spark Thrift Server via beeline. Please correct me, if I am wrong.
You you to be able to connect to Spark Thrift server via beeline, you need to make sure you are providing the correct hostname and portnumber in the JDBC URL you are using in the beeline.
For example:
jdbc:hive2://host:portnumber/
Here, the "host" will be the Hostname of the server where Spark Thriftserver is running.
Let us say it is running on abc.soumya.com.
The default portnumber for Spark Thriftserver is 10000.
But this portnumber can be configured to something else as well. You need to find the correct portnumber.
Thus, you connect string would look like below:
jdbc:hive2://abc.soumya.com:10000/
You can refer the below link for more information on this:
https://spark.apache.org/docs/latest/sql-distributed-sql-engine.html
Created 10-23-2019 07:02 AM
Hi
THANKS FOR THE INFO.
ISSUE RESOLVE .
Created 10-23-2019 08:27 AM
Good to hear that can you share what solution worked for you this way other who encounter the same problem can quickly resolve it.
Thats what we csll community contribution
Happy hadooping
Created 10-23-2019 11:34 PM
Hi @soumya,
Can you please confirm exactly how the issue was resolved.
Please do accept the answer which helped you.
Created 10-24-2019 12:55 AM
actually in oracle Database previously am not entry my user id and password
after entry the same connection is established. It's a minor issue
thanks for your support.