03-04-2015 02:46 PM
I have installed CDH5 on my ubuntu 14.04 successfully.
but for spark only history server is running . Master and worker is not running .
is it required to manually start these services ?
please provide help
06-20-2016 03:45 AM - edited 06-20-2016 03:46 AM
Im happy to set up a separate post, but was hoping you could pick this up here. Im having a similar problem with Spark and Hue on CDM.
I have a running CDM 5.7.1 cluster on Ubuntu 14.04, with all services working fine (apart from spark and impala).
It apears that the spark hostory servers and gateways are installed, but I cant activate Spark in Standalone or Spark on Yarn. In the parcels section I am getting errors across a number of services:
Having checked, this appears to mean that there isnt a Ubuntu Trusty version of the above parcels.
Can you confirm if this is the case.
If so, can I install the components via apt-get:
sudo apt-get install spark-core spark-master spark-worker spark-history-server spark-python
as described in this link for CDM 5.4.x:
Any guidance on this would be appreciated.
After installing spark, I'd like to activate the Hue Spark Notebook, but can see that in Hue app_blacklist is set to:
['spark', 'zookeeper', 'security']
I have removed spark and zookeeper from the app_blacklist leaving 'security' and have restarted the Hue service, and refreshed Hue web UI, I can see only 'security' in the Hue.ini dump but still dont have any spark notebook available. This may be due to the dependency on Spark Parcels being installed.
If I have to re-install the cluster onto Linux RedHat to activate the Spark Parcels that could be a possibility, but I'd prefer to get everything working on ubuntu 14.04 first if possible.
Any guidance on which route to take would be appreciated.