Thank you! That definitely helps. ... View more

Thank you for the quick response, I really appreciate helping me clear my questions.    The answer was exactly what I was looking for. It is automated and users cannot control the elements of classpath.txt file.    Pardon my naive question, but can it pose a problem having different versions of same dependencies on classpath?    Example:   09:39:34 $ cat /etc/spark/conf/classpath.txt | grep jersey-server /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p886.563/jars/jersey-server-1.9.jar /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p886.563/jars/jersey-server-1.14.jar   ... View more

Thank you for your response Wilfred. It sure helps me. However, my question was more towards understanding how classpath.txt file mentioned below is created? Does CM create this file on all nodes, is it something we can configure through CM?   08:42:43 $ ll /etc/spark/conf/ total 60 drwxr-xr-x 3 root root 4096   Aug 25 12:28 ./ drwxr-xr-x 3 root root 4096   Aug 25 12:28 ../ -rw-r--r-- 1 root root 29228 Aug 25 12:28 classpath.txt -rw-r--r-- 1 root root 21       Aug 25 12:28 __cloudera_generation__ -rw-r--r-- 1 root root 550     Aug 25 12:28 log4j.properties -rw-r--r-- 1 root root 800     Aug 25 12:28 spark-defaults.conf -rw-r--r-- 1 root root 1122   Aug 25 12:28 spark-env.sh drwxr-xr-x 2 root root 4096   Aug 25 12:28 yarn-conf/       ... View more

I'm having a similar issue but with a different stack trace. Were you able to resolve this problem?  ... View more