Reply
Explorer
Posts: 15
Registered: ‎06-06-2017

Re: Spark

The issue remains same, not yet have any solution, actually proceeding with other options. sorry.

Explorer
Posts: 8
Registered: ‎03-23-2016

Re: Spark

Any news or what is your current workaround to connect spark with impala? 

New Contributor
Posts: 1
Registered: ‎08-17-2017

Re: Spark & Impala Exception in Yarn Cluster

We are having the same problem.

Explorer
Posts: 23
Registered: ‎01-14-2015

Re: Spark & Impala Exception in Yarn Cluster

After investing some time into this I am finally able to connect to Kerberized Impala environment using Spark and Impala JDBC. I have uploaded my code to github. You can check it out and get hints from here :-

https://github.com/morfious902002/impala-spark-jdbc-kerberos

Explorer
Posts: 23
Registered: ‎01-14-2015

Re: Spark & Impala Exception in Yarn Cluster

[ Edited ]
 
New Contributor
Posts: 1
Registered: ‎12-19-2018

Re: Spark & Impala Exception in Yarn Cluster

In Spark 1.6, I had the same issue and resolved by creating the connection as below code:

 

protected def getImpalaConnection(impalaJdbcDriver: String, impalaJdbcUrl: String): Connection = {
if (impalaJdbcDriver.length() == 0) return null
try {
Class.forName(impalaJdbcDriver).newInstance
UserGroupInformation.getLoginUser.doAs(
new PrivilegedAction[Connection] {
override def run(): Connection = DriverManager.getConnection(impalaJdbcUrl)
}
)
} catch {
case se: SQLException => println(se.printStackTrace())
throw se
case e: Exception => println(e.printStackTrace())
throw e
}
}

 

But same code failed in Spark2, still trying to resolve the issue.

Announcements