I had the same problem with a fat scala jar that no longer worked after an upgrade to Cloudera 6.3.3. This did the trick for me: Build a thin jar instead by using the "provided" tag in the build.sbt "org.apache.spark" %% "spark-core" % "2.4.0" % "provided", "org.apache.spark" %% "spark-sql" % "2.4.0" % "provided", "org.apache.spark" %% "spark-hive" % "2.4.0" % "provided", "commons-httpclient" % "commons-httpclient" % "3.1",  The httpclient jar had to be added and I also had to force utf-8 encoding. ... View more