Receiving java.lang.AbstractMethodError on jackson libraries

[#1]

I am facing class incompatibility issue when cascading-hive job is run using TEZ on HDP Sandbox version 2.4. jackson-mapper-asl.jar is in both hive–exec.jar(bundled) and Hadoop lib path. Hadoop uses version 1.9.13 and hive-exec seems to use either 1.9.10 or lower. Currently stopped using Timeline Service for TEZ to overcome the problem. any better alternative?

java.lang.AbstractMethodError: org.codehaus.jackson.map.AnnotationIntrospector.findSerializer(Lorg/codehaus/jackson/map/introspect/Annotated;)Ljava/lang/Object; at org.codehaus.jackson.map.ser.BasicSerializerFactory.findSerializerFromAnnotation(BasicSerializerFactory.java:366) at org.codehaus.jackson.map.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:252) at org.codehaus.jackson.map.ser.StdSerializerProvider._createUntypedSerializer(StdSerializerProvider.java:782) at org.codehaus.jackson.map.ser.StdSerializerProvider._createAndCacheUntypedSerializer(StdSerializerProvider.java:735) at org.codehaus.jackson.map.ser.StdSerializerProvider.findValueSerializer(StdSerializerProvider.java:344) at org.codehaus.jackson.map.ser.StdSerializerProvider.findTypedValueSerializer(StdSerializerProvider.java:420) at org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:601) at org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256) at org.codehaus.jackson.map.ObjectMapper.writeValue(ObjectMapper.java:1613) at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$EntityLogFD.writeEntities(TimelineClientImpl.java:924) at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$LogFDsCache.createSummaryFDAndWrite(TimelineClientImpl.java:1438) at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$LogFDsCache.writeSummmaryEntityLogs(TimelineClientImpl.java:1418) at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$LogFDsCache.writeSummaryEntityLogs(TimelineClientImpl.java:1397) at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:823) at org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService.handleEvents(ATSV15HistoryLoggingService.java:375) at org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService.access$900(ATSV15HistoryLoggingService.java:53) at org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService$1.run(ATSV15HistoryLoggingService.java:195) at java.lang.Thread.run(Thread.java:745)

[#2 · Accepted solution]

@Srinivas Badveli did you try HADOOP_USER_CLASSPATH_FIRST (mapreduce.job.user.classpath.first) which actually allow prepend user app classpath first so the desired lib can be picked up.

[#3]

Know this is old but I am seeing the same issue. Just curious as to whether HADOOP_USER_CLASSPATH_FIRST solved the problem? Assume so due to the thumbsup on the comment but you mentioned mapreduce.job.user.classpath.first, is that setting picked up by TEZ jobs as well and not only MR? We're running version 0.7.0 of tez and afaik no similar flag exist there (in 0.9.0 there's tez.user.classpath.first).

In our case we do not get this AbstractMethodError on every run so it seems that wrong class version is only picked sporadically.

[#4]

Hi @Thomas Enqvist

Were you able to solve this sporadic issue?