Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark SQL?

Solved Go to solution

Spark SQL?

Do we need to know Spark SQL for the CCA Spark and Hadoop certi?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Spark SQL?

Expert Contributor

Support for Spark SQL is being added into CDH5.5  As of today, the exam is running on CDH5.3.2

So the answer is "not yet", but that will almost certainly change in the near future.

 

Watch the Cloudera website:  http://www.cloudera.com/training/certification/cca-spark.html

The list of required skills should give you knowledge of what technologies you will need to know.

3 REPLIES 3

Re: Spark SQL?

Expert Contributor

Support for Spark SQL is being added into CDH5.5  As of today, the exam is running on CDH5.3.2

So the answer is "not yet", but that will almost certainly change in the near future.

 

Watch the Cloudera website:  http://www.cloudera.com/training/certification/cca-spark.html

The list of required skills should give you knowledge of what technologies you will need to know.

Re: Spark SQL?

Explorer

CDH 5.3.0 ships with Spark 1.2.0 which in turn ships with support for Spark SQL. So I guess all CDH >= 5.3.0 must support Spark SQL. Unless CDH explicitly comes without Spark SQL support...

 

See http://spark.apache.org/docs/1.2.1/sql-programming-guide.html

Highlighted

Re: Spark SQL?

Super Collaborator
That’s not correct. Please see the release notes
http://www.cloudera.com/documentation/enterprise/5-3-x/topics/cdh_rn_spark_ki.html

SparkSQL just exited alpha and is far from stable. As such, SparkSQL is
currently considered a “preview” in CDH. We love it and we’re
dedicating a lot of engineering resources to bring it to our standards
but as I’m sure you’re aware, it’s mainly Scala (pyspark lags),
it’s very buggy, it causes all kinds of havoc (esp. with Hive)….the
list goes on.

Once we get it running at scale, we’ll support it fully in our
distribution and we’ll test it. But today, it’s just not ready.


Don't have an account?
Coming from Hortonworks? Activate your account here