Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

pyspark issue


@hema moger

This would be a sample code to covert csv to json using pyspark.

df ="CSV").option("header","true").load("file:///tmp/sample.csv")

Hope this helps.

@hema moger, Do accept this answer and close this thread if it helped in addressing your query.

@hema moger

In spark1.6, you can use databricks custom csv formatter to load csv into a data frame and write it to a json. You can read this readme to achieve that

In spark2+, spark itself providing a csv loader to create a data frame and write it to a whatever format (json, parquet and orc) you want

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.