Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

HDPCD: Spark (Python) - Create a SparkContext?

New Contributor

I took the HDPCD: Spark (Python) exam yesterday. I had to save my solution code in text files. The exam guide said that these could should 'run in the shell'. I was unsure, however, whether this refers to the Linux shell or the (Py-)Spark shell. In the first case, it is necessary to add initializations:

#!/usr/bin/env python
from pyspark import SparkContext, HiveContext
sc = SparkContext()
sqlContext = HiveContext(sc)

Is it necessary to add these lines to every solution in the HDPCD: Spark exam?

0 REPLIES 0