Hi Shilpa,

I am assuming that you selected the Spark service from the base CDH 5.9 parcels/packages and didn't fetch is separate and therefor are running a standalone Spark cluster alongside the Hadoop cluster.

In any case, the Spark Gateway, as well as the other Gateway's (HDFS, YARN, HBase, etc.), only install the libraries, binaries, and configuration files required to use the command line tools.  In this case it is spark-submit and spark-shell (maybe the pyspark files as well, not 100% sure on it).  There is not service to start and stop or monitor.  That is why you can't start it and why it is gray instead of green.

In the case of the above assumption, it automatically sets up the tools to run in YARN mode.  This means that the spark application runs in YARN instead of a standalone Spark cluster.  Find the ResourceManager UI.  You will find the application listed there and can access the AM and worker logs.  You will be direct to the Spark History server UI (so you could go directly here if you wanted but it is good to get used to the RM interface).