error when spark 1.5 load data from relation database(oracle) using dataframe

[#1]

hi,I upgrade cdh from 5.4.8 to 5.5 ,and now the

 spark 1.5 cann't load data from oracle , it appears

 java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.Module$SetupContext.setClassIntrospector(Lcom/fasterxml/jack son/databind/introspect/ClassIntrospector;)V 

I replace the jar jackson-module-scala_2.10-2.2.3 with jackson-module-scala_2.10-2.4.4 ,it doesn't work

[#2]

Could you share your script, and also let us know if your spark application jar bundles any form of custom jackson libraries, or any dependencies in it that transiently do so? It would be helpful for us to have the output of jar tf your-jar.jar and/or javap -classpath your-jar.jar

'com.fasterxml.jackson.databind.Module$SetupContext'.

The missing method has been present in jackson-databind since 2.2, so the issue is that there's a version older than 2.2 on your Spark classpath, coming in from somewhere unintentionally.

I'm able to run a simple RDBMS query on a clean 5.5.0 environment's spark-shell, and the below is my classpath.txt analysis:

~> for jr in $(grep jackson /etc/spark/conf/classpath.txt); do jar tf $jr | grep databind/Module.class && javap -classpath $jr 'com.fasterxml.jackson.databind.Module$SetupContext' | grep setClassIntrospector; done
com/fasterxml/jackson/databind/Module.class
public abstract void setClassIntrospector(com.fasterxml.jackson.databind.introspect.ClassIntrospector);
com/fasterxml/jackson/databind/Module.class
public abstract void setClassIntrospector(com.fasterxml.jackson.databind.introspect.ClassIntrospector);
com/fasterxml/jackson/databind/Module.class
public abstract void setClassIntrospector(com.fasterxml.jackson.databind.introspect.ClassIntrospector);

[#3]

hi,thank you for you prompt reply. I've downgraded it to 5.4.8.   (spark 1.3, it works well),   I was wondering if you could wait for a while  until I  turn it back  and try again.  

[#4]

I think I understand what causes the problem, I use phoenix (for cdh 5.5) ,and in its pom.xml,it integrates  with spark 1.4.

in /etc/spark/conf/spark-default.conf

spark.executor.extraClassPath=/opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-1.2.0-client.jar
spark.driver.extraClassPath=/opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-1.2.0-client.jar

command

./spark-shell  --jars /home/app/product/client_1/jdbc/ojdbc6.jar

scala> val jdbcDF = sqlContext.read.format("jdbc").options(Map("url" -> "jdbc:oracle:thin:test/test@//192.168.1.10:1521/orcl","dbtable" -> "store","driver" -> "oracle.jdbc.OracleDriver")).load()
scala > jdbcDF.show()

java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.Module$SetupContext.setClassIntrospector(Lcom/fasterxml/jackson/databind/introspect/ClassIntrospector;)V
        at com.fasterxml.jackson.module.scala.introspect.ScalaClassIntrospectorModule$$anonfun$1.apply(ScalaClassIntrospector.scala:32)
....

[#5]

So could you tell me your solution? Rebuild the phoenix with spark 1.5? Or notify the phoenix the problem and wait for official answer?

[#6]

I got this problem too ,I wish to know how did you solve it

[#7]

remove 

spark.executor.extraClassPath=/opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-1.2.0-client.jar
spark.driver.extraClassPath=/opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-1.2.0-client.jar

from /etc/spark/conf/spark-default.conf

or  downgrade cdh from 5.5 to 5.4.8  . 

[#8]

I hope to use the apache phoenix plugin to get data from hbase phoenix

[#9]

I hope to use the apache phoenix plugin to get data from hbase phoenix,so if i remove them from the conf file ,will the plugin work?

[#10]

I faced the same problem with CDH 5.5.1 + Phoenix 4.5.2, both installed with Parcels. I resolved (or worked around?) by prepending jackson-databind-2.2.3.jar in spark-submit jar list.

spark-submit --master yarn --deploy-mode client ... --jars /opt/cloudera/parcels/CDH/jars/jackson-databind-2.2.3.jar,<my jars, phoenix jar>

(I can only run my spark program in client mode. Not sure whether this workaround is usable in other modes)

There are several jackson-databind*jar in CDH Parcel jar directory. The reason I use jackson-databind-2.2.3.jar, is that there is a jackson-module-scala_2.10-2.2.3.jar. Therefore, I sticked to version 2.2.3.