03-25-2016 10:50 AM
Can we launch Spark Application without using spark-submit command . as there any other method means using rest api or some other framework . so we can launch spark application from outside of cluster.
03-27-2016 05:50 PM
Spark job server is REST API.
And spark job server should be integrated in hue.
This component is still beta release in hue.
03-27-2016 09:52 PM - edited 03-27-2016 09:53 PM
According to the following page, hue supported spark notebook(frontend of livy server) since 3.8.
CDH hue might have become 3.9 since 5.5.0
so, CDH 5.5.1 seems to have livy server.
However, even if cdh 5.6.0, spark notebook seems to be still experimental.
If you use cdh hue, livy server can be booted in foreground process with the following command.
# sudo -u hue /usr/lib/hue/build/env/bin/hue livy_server