Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

write a file to HDFS using Spark

avatar

hi,

i am able to read a file from HDFS in Spark e using sc.textFile("path")

Can smeone help me How to write a file in HDFS using spark

1 ACCEPTED SOLUTION

avatar
Contributor

Hi,

 

According documentation, you may try the saveAsTextFile method:

http://spark.apache.org/docs/latest/programming-guide.html

saveAsTextFile(path)Write the elements of the dataset as a text file (or set of text files) in a given directory in the local filesystem, HDFS or any other Hadoop-supported file system. Spark will call toString on each element to convert it to a line of text in the file.

View solution in original post

2 REPLIES 2

avatar
Contributor

Hi,

 

According documentation, you may try the saveAsTextFile method:

http://spark.apache.org/docs/latest/programming-guide.html

saveAsTextFile(path)Write the elements of the dataset as a text file (or set of text files) in a given directory in the local filesystem, HDFS or any other Hadoop-supported file system. Spark will call toString on each element to convert it to a line of text in the file.

avatar

Thanks