question Re: 3rd jackson-core-*.jar on HDP install in Support Questions

3rd jackson-core-*.jar on HDP install

sunile_manjee — Sat, 09 Jan 2016 02:05:51 GMT

I am finding many libraries part of HDP base install are added to class path like jackson-core-2.2.3.jar. However these libraries do not come with the vanilla (non hdp) install. Does anyone know why and how these libraries are used? If new version of the library exist, HDP may force to specify classpath for each application running with different jar. Is there a possible work around where HDP separate core hadoop libraries classpth from non core libraries (like jackson..*)?

Re: 3rd jackson-core-*.jar on HDP install

nsabharwal — Sat, 09 Jan 2016 09:06:08 GMT

@Sunile Manjee

Interesting question 🙂

Core Jackson abstractions, basic JSON streaming API implementation

http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core/2.2.3

Re: 3rd jackson-core-*.jar on HDP install

sunile_manjee — Wed, 03 Feb 2016 12:17:03 GMT

There is no solution for this at the moment. I have informed @Tim Hall. Work around is using maven shading.

Re: 3rd jackson-core-*.jar on HDP install

magnus — Wed, 09 Mar 2016 22:26:05 GMT

The presence of this dependency is causing issues for users using both HDP and CDH it seems; https://github.com/druid-io/druid/blob/master/docs/content/operations/other-hadoop.md#cdh

If its not a necessary component and the work to extract it is relatively small it would save a lot of headache if it can be extracted from core libraries

Re: 3rd jackson-core-*.jar on HDP install

aervits — Thu, 10 Mar 2016 00:13:58 GMT

Use maven exclude for this jar as a workaround when you deploy jars.

Re: 3rd jackson-core-*.jar on HDP install

magnus — Thu, 10 Mar 2016 03:15:07 GMT

@Artem Ervits Care to elaborate? Which jar are you talking about? when we deploy Hadoop, HDP, Druid or our Job? Excluding the dependency in Druid is the headache since the versions conflict. if I understood this question correctly @Sunile Manjee suggests separating the Jackson dependencies from the core Hadoop libraries which should solve this and future version conflicts for Jackson

Re: 3rd jackson-core-*.jar on HDP install

aervits — Thu, 10 Mar 2016 06:11:35 GMT

You can either parse your classpatg and remove the dependency programmatically or if you are getting an error when you deploy an application, use <exclude> tag ofaven to remove a conflicting, usually an older version of Jackson from your build. In my case it was the latter, maven was pulling an older version of Jackson core and in classpatg it was appearing before the one required by my mapreduce program.

Re: 3rd jackson-core-*.jar on HDP install

prabhu_sekhar — Wed, 02 Aug 2017 19:27:06 GMT

is this fixed? We are facing same issue with json-smart.jar.. My application uses 2.1 version, whereas hadoop 2.6 has 1.1.1 version in the classpath..