Apache phoenix 4.5.1 with spark 1.3.1 Classnotfoundexception

[#1]

We are running spark streaming to get the feed from Kafka. Now we are trying to use Phoenix JDBC to extract some data from HBASE. When I run the code in local its running fine without any issues, but when I run it using yarn-cluster it throws below exception. Using CDH 5.4.4.

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)

Below is the code snippet:

def check(arg0:String,arg1:String)
{
  val query = "query"

        Class.forName("org.apache.phoenix.jdbc.PhoenixDriver");
    var conn =  DriverManager.getConnection("jdbc:phoenix:IP:zkport:/znode");
        val statement = conn.createStatement()
        val resultSet = statement.executeQuery(query)
        var out="";
        while(resultSet.next())
        {
          out=resultSet.getString("..")
        }
}

And the SBT dependency added is

libraryDependencies += "org.apache.phoenix" % "phoenix-core" % "4.5.2-cdh5.4.5"

I manually checked for the missing class, and it is there in phoenix-core jar. What is the reason behind yarn/spark throwing an exception. I tried adding it as a separate classpath that doesn't work either. Any workaround for this.

[#2]

Hello,

           are you using the Phonex Clab ? What  is the version of it ?

Kind Regards

[#3]

Yes, 4.5.2-cdh5.4.5 is the one in my dependency. Got that from cloudera repo https://repository.cloudera.com/cloudera/cloudera-repos/org/apache/phoenix/phoenix-core/

[#4 · Accepted solution]

Adding phoenix-core to classpath.txt worked. But a weird issue.

[#5]

I am getting the same error when I try to load a phoenix table into Spark DataFrame. Have you come across same issue with dataframes?

[#6]

Not with dataframes but i think it is a collon issue with phoenix+spark streaming. You can try adding the jars to classpath and execute