Thank you Benjamin.. This has fixed the issue.    I would take a note on upgrade point.    thanks Romesh ... View more

Thank you ....   this has not been setup by that command.    [root@sxd-cdh-nn sbin]# /usr/sbin/alternatives --set spark-submit "/opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/bin/spark2-submit" /opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/bin/spark2-submit has not been configured as an alternative for spark-submit [root@sxd-cdh-nn sbin]# /usr/sbin/alternatives --display spark-submit spark-submit - status is auto. link currently points to /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/bin/spark-submit /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/bin/spark-submit - priority 10 Current `best' version is /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/bin/spark-submit. [root@sxd-cdh-nn sbin]# /usr/sbin/alternatives --display spark2-submit spark2-submit - status is auto. link currently points to /opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/bin/spark2-submit /opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/bin/spark2-submit - priority 10 Current `best' version is /opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/bin/spark2-submit. [root@sxd-cdh-nn sbin]#   we did not pre-configured Spark2 while installing CDH and now we have installed spark 2.2 via parcels method.    thanks Romesh  ... View more