Member since
β09-16-2015
4
Posts
1
Kudos Received
0
Solutions
β02-25-2016
10:07 AM
Hi zlpmichelle, This problem came from the fact that I wasn't using a CDH artifact on my Maven dependency. If you package guava 16.0.1 into your jar, you still get this problem? It is still a bit obscure for me how exactly the classpath.txt works and why it mixes several versions of the same API π
... View more
β09-22-2015
03:45 AM
Hi, I'm creating an uber jar containing all application specific libs and dependencies. However my problem comes from a conflict of libs on classpath.txt (it contains conflicting versions of jackson - 2.2.3 and 2.3.1). My application uses jackson as 2.3.1 as dependency but somehow the wrong version of jackson on the classpath is used. My idea was to modify the classpath.txt and deploy it on all worker nodes. Though this file is used only by Gateway nodes. i'm a bit puzzled with this lib conflict since in general, the classpath shound't contain several versions of the same lib. Shouldn't this be better handled on the classpath.txt?
... View more
β09-16-2015
06:34 AM
In the case that the classpath obtained from classpath.txt is given only on the gateway machine, and the worker nodes will use the default configuration (so, no classpath.txt), how the classpath will be synchronized on the whole application? Should I set all worker nodes as gateways then ? o_O AndrΓ©
... View more
β09-16-2015
06:21 AM
1 Kudo
Actually I think i got an issue related to the fact that classpath.txt contains multiple versions of the same jar: The issue is related to this jira : https://issues.apache.org/jira/browse/SPARK-8332 And on /etc/spark/conf/classpath.txt : ----------------------------- cat /etc/spark/conf/classpath.txt | grep jackson /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/jackson-annotations-2.2.3.jar /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/jackson-core-2.2.3.jar /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/jackson-databind-2.2.3.jar /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/jackson-annotations-2.3.0.jar /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/jackson-core-2.3.1.jar /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/jackson-databind-2.3.1.jar ----------------------------- Somehow the classloader is pointing to the version 2.2.3 of jackson, where the method handledType() of the class BigDecimalDeserializer does not exist. Similar errors may appears for jersey as well since the api changed a bit inbetween those versions. Is that a way to solve this kind of issue in a proper way?
... View more