question Re: write a file to HDFS using Spark in Archives of Support Questions (Read Only)

write a file to HDFS using Spark

manish_sharma786 — Fri, 16 Sep 2022 09:37:47 GMT

hi,

i am able to read a file from HDFS in Spark e using sc.textFile("path")

Can smeone help me How to write a file in HDFS using spark

Re: write a file to HDFS using Spark

Grg — Fri, 14 Aug 2015 14:45:15 GMT

Hi,

According documentation, you may try the saveAsTextFile method:

http://spark.apache.org/docs/latest/programming-guide.html

saveAsTextFile(path)

Write the elements of the dataset as a text file (or set of text files) in a given directory in the local filesystem, HDFS or any other Hadoop-supported file system. Spark will call toString on each element to convert it to a line of text in the file.

Re: write a file to HDFS using Spark

manish_sharma786 — Wed, 17 Feb 2016 05:58:21 GMT

Thanks