Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Shading of Akka Libraries Breaking CDSW Sessions

Shading of Akka Libraries Breaking CDSW Sessions


Importing a jar with shaded Akka and Play libraries prevents the CDSW Session from lauching. Sadly, the only error message I get is that in the image below (as the terminal never even gets off the ground.)












These libraries were initially shaded as our internal versions of Akka and Play appear to conflict with some Toree dependencies that CDSW is backended by.


spark-defaults.conf looks like:


spark.driver.extraClassPath /home/cdsw/shaded.jar
spark.executor.extraClassPath ./shaded.jar
spark.jars /home/cdsw/shaded.jar

 The SBT ShadeRules used to create the shaded libs:


assemblyShadeRules in assembly := Seq(
      ShadeRule.rename("play.api.**" -> "internalplay.@1").inAll,
      ShadeRule.rename("akka.**" -> "internalakka.@1").inAll


I can't fathom why our shading would be breaking anything in the session itself. Any ideas how to resolve this issue?


Edit: If anyone knows the play/akka depedency versions for CDSW, that would also be hugely helpful. Could bring our application into fold with what is installed here and hopefully avoid these conficts. We're using play 2.5.9.

Don't have an account?
Coming from Hortonworks? Activate your account here