Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Error while connecting to hive from Spark using jdbc connection string

avatar
Contributor

I am trying to connect to Hive from Spark inside the map function like below

String driver = "org.apache.hive.jdbc.HiveDriver"; Class.forName(driver); Connection con_con1 = DriverManager .getConnection( "jdbc:hive2://server1.net:10001/default;principal=hive/server1.net@abc.xyz.NET;ssl=false;transportMode=http;httpPath=cliservice" , "username", "password");

But i am getting error.

javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

1 ACCEPTED SOLUTION

avatar
Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
7 REPLIES 7

avatar
Rising Star

Looks like the credential issue. Check the credentials entered are correct.

avatar
Contributor

The credentials are correct. I am using the same connection string from edge node and it works fine, but not from the spark program.

avatar
Super Collaborator

I don't know if this applies to your situation, but there is a Java issue that will cause that type of exception with Kerberos 1.8.1 or higher. If your error is related to this, for a renewable ticket run kinit -R after initially running kinit. There are some additional suggestions at https://community.hortonworks.com/articles/4755/common-kerberos-errors-and-solutions.html.

avatar
Rising Star

Your cluster is kerberized and this issue looks like kerberos ticket issue.

you dont have a valid kerberos ticket. to run this command

first get a valid ticket for your user to run this commad.

To get a valid ticket type

#kinit

then it will ask for kerberos password.

after you enter password

#klist

it will display kerberos ticket for your user.

now try running command to connect to hive.

avatar
Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Master Guru

Why connect to hive via JDBC? Use SparkSQL with the HiveContext and have full access to all the HiveTables. This is optimized in spark and very fast.

https://spark.apache.org/docs/1.6.0/sql-programming-guide.html

avatar
Contributor

Yes I agree but I am trying to run hive query inside the map method. While using hive context, I am getting error that class is not serializable.