Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

avatar
Explorer

We are running spark streaming to get the feed from Kafka. Now we are trying to use Phoenix JDBC to extract some data from HBASE. When I run the code in local its running fine without any issues, but when I run it using yarn-cluster it throws below exception. Using CDH 5.4.4.

 

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)

 

Below is the code snippet:

 

def check(arg0:String,arg1:String)
{
  val query = "query"

        Class.forName("org.apache.phoenix.jdbc.PhoenixDriver");
    var conn =  DriverManager.getConnection("jdbc:phoenix:IP:zkport:/znode");
        val statement = conn.createStatement()
        val resultSet = statement.executeQuery(query)
        var out="";
        while(resultSet.next())
        {
          out=resultSet.getString("..")
        }
}

 

And the SBT dependency added is

 

libraryDependencies += "org.apache.phoenix" % "phoenix-core" % "4.5.2-cdh5.4.5"

 

I manually checked for the missing class, and it is there in phoenix-core jar. What is the reason behind yarn/spark throwing an exception. I tried adding it as a separate classpath that doesn't work either. Any workaround for this.

 

1 ACCEPTED SOLUTION

avatar
Explorer

Adding phoenix-core to classpath.txt worked. But a weird issue.

View solution in original post

5 REPLIES 5

avatar
Contributor

Hello,

           are you using the Phonex Clab ? What  is the version of it ?

 

 

Kind Regards

avatar
Explorer

Yes, 4.5.2-cdh5.4.5 is the one in my dependency. Got that from cloudera repo https://repository.cloudera.com/cloudera/cloudera-repos/org/apache/phoenix/phoenix-core/

avatar
Explorer

Adding phoenix-core to classpath.txt worked. But a weird issue.

avatar
New Contributor

I am getting the same error when I try to load a phoenix table into Spark DataFrame. Have you come across same issue with dataframes?

avatar
Explorer

Not with dataframes but i think it is a collon issue with phoenix+spark streaming. You can try adding the jars to classpath and execute