Support Questions
Find answers, ask questions, and share your expertise

Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

Solved Go to solution

Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

New Contributor

We are running spark streaming to get the feed from Kafka. Now we are trying to use Phoenix JDBC to extract some data from HBASE. When I run the code in local its running fine without any issues, but when I run it using yarn-cluster it throws below exception. Using CDH 5.4.4.

 

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)

 

Below is the code snippet:

 

def check(arg0:String,arg1:String)
{
  val query = "query"

        Class.forName("org.apache.phoenix.jdbc.PhoenixDriver");
    var conn =  DriverManager.getConnection("jdbc:phoenix:IP:zkport:/znode");
        val statement = conn.createStatement()
        val resultSet = statement.executeQuery(query)
        var out="";
        while(resultSet.next())
        {
          out=resultSet.getString("..")
        }
}

 

And the SBT dependency added is

 

libraryDependencies += "org.apache.phoenix" % "phoenix-core" % "4.5.2-cdh5.4.5"

 

I manually checked for the missing class, and it is there in phoenix-core jar. What is the reason behind yarn/spark throwing an exception. I tried adding it as a separate classpath that doesn't work either. Any workaround for this.

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

New Contributor

Adding phoenix-core to classpath.txt worked. But a weird issue.

View solution in original post

5 REPLIES 5

Re: Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

Explorer

Hello,

           are you using the Phonex Clab ? What  is the version of it ?

 

 

Kind Regards

Re: Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

New Contributor

Yes, 4.5.2-cdh5.4.5 is the one in my dependency. Got that from cloudera repo https://repository.cloudera.com/cloudera/cloudera-repos/org/apache/phoenix/phoenix-core/

Re: Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

New Contributor

Adding phoenix-core to classpath.txt worked. But a weird issue.

View solution in original post

Re: Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

New Contributor

I am getting the same error when I try to load a phoenix table into Spark DataFrame. Have you come across same issue with dataframes?

Re: Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

New Contributor

Not with dataframes but i think it is a collon issue with phoenix+spark streaming. You can try adding the jars to classpath and execute