- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
write a file to HDFS using Spark
- Labels:
-
Apache Spark
-
HDFS
Created on ‎08-13-2015 10:26 PM - edited ‎09-16-2022 02:37 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi,
i am able to read a file from HDFS in Spark e using sc.textFile("path")
Can smeone help me How to write a file in HDFS using spark
Created ‎08-14-2015 07:45 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
According documentation, you may try the saveAsTextFile method:
http://spark.apache.org/docs/latest/programming-guide.html
saveAsTextFile(path) | Write the elements of the dataset as a text file (or set of text files) in a given directory in the local filesystem, HDFS or any other Hadoop-supported file system. Spark will call toString on each element to convert it to a line of text in the file. |
Created ‎08-14-2015 07:45 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
According documentation, you may try the saveAsTextFile method:
http://spark.apache.org/docs/latest/programming-guide.html
saveAsTextFile(path) | Write the elements of the dataset as a text file (or set of text files) in a given directory in the local filesystem, HDFS or any other Hadoop-supported file system. Spark will call toString on each element to convert it to a line of text in the file. |
Created ‎02-16-2016 09:58 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks
