Hello,
I've noticed a version mismatch in the cloudera doc for the spark2 installation.
From this url ; https://www.cloudera.com/documentation/spark2/latest/topics/spark2_packaging.html#packaging
The CSD version (cloudera1) isn't the latest and then fails when installed along with the "cloudera2" parcel.
The exact error I got during spark2 startup : The CSD version (2.0.0.cloudera1) is not compatible with the current installed version.
For thos who encounter the same issue here is the workaround.
Simply get the latest CSD from here : http://archive.cloudera.com/spark2/csd/SPARK2_ON_YARN-2.0.0.cloudera2.jar
Remove the old cloudera1 CSD, restart cld manager and reinstall spark2.
Cheers!