Support Questions

Find answers, ask questions, and share your expertise

Which version of spark is required for certification

New Contributor

I saw in one of the certification related document about spark 1.6 being mentioned for certification. Do we have to learn that version of spark given that we have latest 2.2.0 version released.

2 REPLIES 2

@rohith ak

HDPCD Spark Certification uses the following versions:

HDP 2.4.0

Spark 1.6

Scala 2.10.5

Python 2.7.6 (pyspark)

Details in link below:

https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2017/05/HDCD_Spark_Data_...

While it's ok to practice on a newer version, just keep in mind that there's functionality in 2.x that is not available in 1.6. I strongly suggest that you study and practice on HDP 2.4 (Spark 1.6) to avoid confusing yourself.

You can download the HDP 2.4 sandbox that contains Spark 1.6 from the link below.

https://hortonworks.com/downloads/#sandbox

Scroll down to where it says "Hortonworks Data Platform Archive" and click "Expand" to get the archived versions and the download link for HDP 2.4.

21495-screen-shot-2017-07-31-at-31316-pm.png

I strongly suggest that you study and practice on HDP 2.4 (Spark 1.6) to avoid confusing yourself.

This is far the best advise you can get. I also recommend to train yourself with Spark 1.6. After you pass the certification you can use all the cool things Spark 2.x offers.