Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Upgrading CDH-provided jars

Upgrading CDH-provided jars

New Contributor

I am trying to run a Spark application on CDH 5.7.0.

The application uses typesafe config 1.2.1, but CDH provides 1.0.2 which takes precendence and causes a NoSuchMethodError (getDuration(String, TimeUnit), which is available in typesafe config 1.2.1 but not in 1.0.2).

 

Can I somehow upgrade the CDH typesafe-config jar, or have my jars take precendece over the ones provided by CDH?

 

Thanks!

1 REPLY 1

Re: Upgrading CDH-provided jars

Master Guru
In general, upgrading any base included jar is likely to break your installation. Such a manual version switch is only safe iff the new version (1) has not removed any of the API calls used in the older version, and (2) has not altered the behaviour of the API calls used in the older version. Typically across large version jumps, one or both of these may not not be true, so the change is not compatible to all the callers.

Perhaps you could shade your application's dependency of the higher typesafe version and bundle it into your application binary jar, placing it inside a different namespace so it does not collide with what CDH uses and offers as default. I haven't tested it for something such as typesafe specifically, but it may be worth a try.