Reply
Highlighted
Explorer
Posts: 21
Registered: ‎03-31-2017

Shading of Akka Libraries Breaking CDSW Sessions

[ Edited ]

Importing a jar with shaded Akka and Play libraries prevents the CDSW Session from lauching. Sadly, the only error message I get is that in the image below (as the terminal never even gets off the ground.)

 

err.JPG

 

 

 

 

 

 

 

 

 

These libraries were initially shaded as our internal versions of Akka and Play appear to conflict with some Toree dependencies that CDSW is backended by.

 

spark-defaults.conf looks like:

 

spark.driver.extraClassPath /home/cdsw/shaded.jar
spark.executor.extraClassPath ./shaded.jar
spark.jars /home/cdsw/shaded.jar

 The SBT ShadeRules used to create the shaded libs:

 

assemblyShadeRules in assembly := Seq(
      ShadeRule.rename("play.api.**" -> "internalplay.@1").inAll,
      ShadeRule.rename("akka.**" -> "internalakka.@1").inAll
    )

 

I can't fathom why our shading would be breaking anything in the session itself. Any ideas how to resolve this issue?

 

Edit: If anyone knows the play/akka depedency versions for CDSW, that would also be hugely helpful. Could bring our application into fold with what is installed here and hopefully avoid these conficts. We're using play 2.5.9.

Announcements