3rd jackson-core-*.jar on HDP install

[#1]

I am finding many libraries part of HDP base install are added to class path like jackson-core-2.2.3.jar. However these libraries do not come with the vanilla (non hdp) install. Does anyone know why and how these libraries are used? If new version of the library exist, HDP may force to specify classpath for each application running with different jar. Is there a possible work around where HDP separate core hadoop libraries classpth from non core libraries (like jackson..*)?

[#2]

@Sunile Manjee

Interesting question 🙂

Core Jackson abstractions, basic JSON streaming API implementation

http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core/2.2.3

[#3 · Accepted solution]

There is no solution for this at the moment. I have informed @Tim Hall. Work around is using maven shading.

[#4]

The presence of this dependency is causing issues for users using both HDP and CDH it seems; https://github.com/druid-io/druid/blob/master/docs/content/operations/other-hadoop.md#cdh

If its not a necessary component and the work to extract it is relatively small it would save a lot of headache if it can be extracted from core libraries

[#5]

Use maven exclude for this jar as a workaround when you deploy jars.

[#6]

@Artem Ervits Care to elaborate? Which jar are you talking about? when we deploy Hadoop, HDP, Druid or our Job? Excluding the dependency in Druid is the headache since the versions conflict. if I understood this question correctly @Sunile Manjee suggests separating the Jackson dependencies from the core Hadoop libraries which should solve this and future version conflicts for Jackson

[#7]

You can either parse your classpatg and remove the dependency programmatically or if you are getting an error when you deploy an application, use <exclude> tag ofaven to remove a conflicting, usually an older version of Jackson from your build. In my case it was the latter, maven was pulling an older version of Jackson core and in classpatg it was appearing before the one required by my mapreduce program.

[#8]

is this fixed? We are facing same issue with json-smart.jar.. My application uses 2.1 version, whereas hadoop 2.6 has 1.1.1 version in the classpath..