@VijayM , Couple of questions: 1. Are CDH6 and CDH5 managed by same Cloudera Manager, or you manage yourself? 2. From the setting you applied below: spark.yarn.jars=local:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark/jars/*,local:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark/hive/*,local:/app/bds/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/spark/lib/* It looks like you have both CDH6.2 and CDH5.16 running on the same host, is that right? Any reason you want to do so? As @JosiahGoodson mentioned, spark2 and spark1 are not compatible, you should either have Spark1, or Spark2 jars in the classpath, not both, otherwise they will conflict. Cheers Eric ... View more