This would be a sample code to covert csv to json using pyspark.
df = spark.read.format("CSV").option("header","true").load("file:///tmp/sample.csv") df.repartition(1).toJSON(use_unicode=True).saveAsTextFile("file:///tmp/sample_out")
Hope this helps.
In spark1.6, you can use databricks custom csv formatter to load csv into a data frame and write it to a json. You can read this readme to achieve that
In spark2+, spark itself providing a csv loader to create a data frame and write it to a whatever format (json, parquet and orc) you want