Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

SparkSQL in CDH 5.1.0

Highlighted

SparkSQL in CDH 5.1.0

Expert Contributor

Are there any examples on how to create a Spark application using SparkSQL that comes with CDH 5.1.0 using Hue?

 

Thanks!

6 REPLIES 6
Highlighted

Re: SparkSQL in CDH 5.1.0

Master Collaborator

It is the same as what you might find in the project's docs; it's not any different.

 

https://spark.apache.org/docs/latest/sql-programming-guide.html

Highlighted

Re: SparkSQL in CDH 5.1.0

Expert Contributor

Thanks Sean, I wanted to know what are the options available (apart from spark-submit) to create workflows or schedule the batch jobs in CDH 5.1.0. I was researching the same in hue-users list as well but so far spark-submit was the only option that came up, since Hue/Oozie doesn't support Spark action yet in Hue 3.6.

 

Please let me know if there are other alternatives.

 

 

Highlighted

Re: SparkSQL in CDH 5.1.0

Master Collaborator

Here is how to use Spark from Hue:

 

http://gethue.com/get-started-with-spark-deploy-spark-server-and-compute-pi-from-your-web-browser/

 

It takes a bit of extra manual work to set up the pieces though.

Highlighted

Re: SparkSQL in CDH 5.1.0

Expert Contributor

Yes Sean, the job server is what I've been using with CDH-5.0.3 (Spark 0.9) but it doesn't yet support Spark 1.0 in CDH-5.1.0. In any case, thanks for the info.

 

 

 

 

Highlighted

Re: SparkSQL in CDH 5.1.0

Master Collaborator

Hm, what doesn't work? I haven't tried it recently myself. It could be some fixable error.

Highlighted

Re: SparkSQL in CDH 5.1.0

Expert Contributor

Here is the thread with Evan regarding the issue:

  https://groups.google.com/forum/#!topic/spark-jobserver/jCka16MUN-4

Don't have an account?
Coming from Hortonworks? Activate your account here