Hi, thank you for the reply, but I don't think I can make AWS SDK use the shaded version without recompiling it, which was the use case(correct me if I'm wrong, I'm no expert in dependency management). I couldn't get to AWS services because SDK was dependent on Joda. The basic example I wrote was just a way to test the issue.
... View more
Hi. I'm seeing an issue that looks like: https://community.cloudera.com/t5/Batch-Processing-and-Workflow/CDH-5-3-CLASSPATH-Issue/m-p/23901#M898 On cloudera 5.5.2, I can't submit any job that has joda in any major capacity, for example, if I write this: org.joda.time.format.DateTimeFormatter sdf = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss"); In a jar and submit it to a cluster I get this error: Exception in thread "main" java.lang.NoClassDefFoundError: org/joda/time/format/DateTimeFormat -----------(My code line)---------------- at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) -----------(My code line)---------------- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.lang.ClassNotFoundException: org.joda.time.format.DateTimeFormat at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 9 more Including or excluding joda from the jar submitted makes no difference at all. It wouldn't be a big issue if I didn't need to use other libraries that use joda-time. Like AWS SDK My guess is that, since cloudera itself has aws-sdk jars, but doesn't have joda jars. Joda is probably embeded into another jar with bits missing. I have worked around the issue in my project. But that's something you might want to look into. To clearify, I only have hadoop, yarn(both rm&nm) and hdfs(both dn&nn) installed on mint
... View more