I'm trying to use the Phoenix-Spark2 connector in Zeppelin as described here and having some confusion about dependencies Here is the code I'm running: import org.apache.spark.SparkContext
import org.apache.spark.sql.SQLContext
import org.apache.phoenix.spark._
val sc = new SparkContext("local", "phoenix-test")
val sqlContext = new SQLContext(sc)
val df = sqlContext.load(
"org.apache.phoenix.spark",
Map("table" -> "dz_places",
"zkUrl" -> "driver_node:2181"))
I keep getting ClassNotFound Exceptions so I go and find the related jar and add it to the Zeppelin Spark2 interpreter dependencies. So far I've added these jars: /usr/hdp/current/phoenix-server/lib/phoenix-spark-5.0.0.3.0.0.0-1634.jar
/usr/hdp/current/phoenix-server/lib/phoenix-core-5.0.0.3.0.0.0-1634.jar
/usr/hdp/current/hbase-master/lib/hbase-common-2.0.0.3.0.0.0-1634.jar
/usr/hdp/current/hbase-client/lib/hbase-client-2.0.0.3.0.0.0-1634.jar
/usr/hdp/current/hbase-client/lib/htrace-core-3.2.0-incubating.jar Now I'm seeing this error: java.lang.NoClassDefFoundError: org/apache/tephra/TransactionSystemClient I'm not seeing this jar in any of the Hbase or Phoenix lib folders. What's going on? Why do I need to add all these? Where is this particular class housed? Is there a better way to specify these? Using /usr/hdp/current/hbase-client/lib/*.jar threw an error.
... View more
