Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on 08-13-2015 10:26 PM - edited 09-16-2022 02:37 AM
hi,
i am able to read a file from HDFS in Spark e using sc.textFile("path")
Can smeone help me How to write a file in HDFS using spark
Created 08-14-2015 07:45 AM
Hi,
According documentation, you may try the saveAsTextFile method:
http://spark.apache.org/docs/latest/programming-guide.html
| saveAsTextFile(path) | Write the elements of the dataset as a text file (or set of text files) in a given directory in the local filesystem, HDFS or any other Hadoop-supported file system. Spark will call toString on each element to convert it to a line of text in the file. |
Created 08-14-2015 07:45 AM
Hi,
According documentation, you may try the saveAsTextFile method:
http://spark.apache.org/docs/latest/programming-guide.html
| saveAsTextFile(path) | Write the elements of the dataset as a text file (or set of text files) in a given directory in the local filesystem, HDFS or any other Hadoop-supported file system. Spark will call toString on each element to convert it to a line of text in the file. |
Created 02-16-2016 09:58 PM
Thanks