Support Questions
Find answers, ask questions, and share your expertise

I attempted Spark Certification Exam today and found, I could not write a CSV file though I executed my .py task as below.


spark-submit --packages com:databricks:spark-csv_2.10:1.2.0

I got an error that the class could not be found. My question is how can I write a CSV file using the dataframes like below


Any suggestions on how can I solve the problem of writing a CSV or a TAB file in the certification Exam, I am pretty sure, I failed it, since I could not write.



Any help is greatly appreciated, I am planning to take the exam again soon, but my question is what are my choices, if I get the same error while writing a file, though my inputs are correct...

New Contributor

From spark 2.0.2. You can use without using databricks spark package.


Expert Contributor => line.mkString(","))


Hi, I did the spark certification exam and had the same issue.

We can not download packages while launching spark as the environment is locked from any outside downloads.

--packages pull the spark-csv package from the maven repository.



In python, you are able to use the python csv package that comes as a standard library with the python install.

You do nat have access to Pandas, SKLearn and any package mangers such as pip and easy-install. They are locked out from the certification environment.

Cloudera Employee

See the end of my comment above. If you can copy jar files manually you can use --jars instead of --packages.


Assuming that there are three columns in DF as (col1:Int, col2:String, col3:Float), the CSV file output can be achieved as: => (x.getInt(0) + "," + x.getString(1) + "," + x.getFloat(2))).saveAsTextFile("out.csv")

--- OR --- => (x(0) + "," + x(1) + "," + x(2))).saveAsTextFile("out.csv")

New Contributor
Same problem today how did you solve it? How to write a csv file from hive table without databriks obviously not present in the verisone 1.6 of spark in the exam. I feel fragile I could not complete the exam.