question Re: CCA 175 : spark version 1.6 on exam: in Archives of Support Questions (Read Only)

CCA 175 : spark version 1.6 on exam:

JB0000000000001 — Fri, 16 Sep 2022 12:55:29 GMT

Dear community,

I notice the exam on the CCA175 will have spark version 1.6. One of the main topics of the exam is data analysis using spark SQL. I notice that the functionalities to load a dataframe into a format that can be used to perform sql queries, only exist since spark version >1.6 (e.g. registerTempTable or createorreplacetempview).

ANy thoughts on this? I am surprised that such an outdated version of spark is used for the exam.

Best to all!

Re: CCA 175 : spark version 1.6 on exam:

Former Member — Sat, 03 Mar 2018 19:22:41 GMT

You may be having an incorrect understanding when you say

I notice that the functionalities to load a dataframe into a format that can be used to perform sql queries, only exist since spark version >1.6

Can you expand on it with an example?

Re: CCA 175 : spark version 1.6 on exam:

JB0000000000001 — Mon, 05 Mar 2018 08:06:00 GMT

I was referring to the following which is not available yet in spark 1.6 :

1)create a DF

2)create a table to write direct sql queries on: df.createGlobalTempView("people")

3)query on this table : spark.sql("SELECT * FROM global_temp.people")

But I think what is required for the section "data analysis: use spark sql to interact with the metastore programmatically in your application" is to create a SQL/HiveContext and then query on tables that are already stored in the HIVE metastore. ANy idea if this is correct?

Re: CCA 175 : spark version 1.6 on exam:

Former Member — Mon, 05 Mar 2018 09:59:19 GMT

It would be beneficial for you if you can go through Spark 1.6 documentation fully, in order to understand what is available and what is not. For your questions, see below.

1. sqlContext.createDataFrame(your_query)
    rdd.toDF(schema)

2. df.registerTempTable(table_name)

3. sqlContext.sql(your_query)

sqlContext.read.table(your_hive_table)