Reply
Explorer
Posts: 16
Registered: ‎10-27-2015

the issue of phoenix 4.7 intergration with spark

hi all

     i just tested  phoenix 4.7 against cdh 5.7

    my env is 

    cdh :5.7.2-1.cdh5.7.2.p0.18

    phoenix  :4.7.0-1.clabs_phoenix1.3.0.p0

  i added two line in  the spark-default.conf 

 

 

spark.executor.extraClassPath=/opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-4.7.0-clabs-phoenix1.3.0-client.jar 
spark.driver.extraClassPath=/opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-4.7.0-clabs-phoenix1.3.0-client.jar

  then launched  spark-shell 

 

 

Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_67)
Type in expressions to have them evaluated.
Type :help for more information.
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.Module$SetupContext.setClassIntrospector(Lcom/fasterxml/jackson/databind/introspect/ClassIntrospector;)V

 

Contributor
Posts: 51
Registered: ‎03-04-2015

Re: the issue of phoenix 4.7 intergration with spark

You might have version conflict among multiple copies of jackson-databind included in different components.  Adding "/opt/cloudera/parcels/CDH/jars/jackson-databind-2.2.3.jar" to beginning of the two extraClassPath did it for us.  You might want to try it out.

 

However, that's for spark-submit.  Oozie is whole other can o' worms entirely ...

 

Announcements

Currently incubating in Cloudera Labs:

Envelope
HTrace
Ibis
Impyla
Livy
Oryx
Phoenix
Spark Runner for Beam SDK
Time Series for Spark
YCSB