Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Error saveAsTextFile with pyspark
Labels:
- Labels:
-
Apache Spark
Master Collaborator
Created ‎11-21-2018 01:35 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have this error from pyspark, when I type this, any suggestion?
counts.saveAsTextFile("file:///tmp/JUPYTERR_out7")
Caused by: java.io.IOException: Failed to rename DeprecatedRawLocalFileStatus{path=file:/tmp/JUPYTERR_out7/_temporary/0/task_20181121142733_0009_m_000001/part-00001; isDirectory=false; length=6448; replication=1; blocksize=33554432; modification_time=1542806853000; access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false} to file:/tmp/JUPYTERR_out7/part-00001
2 REPLIES 2
Rising Star
Created ‎11-21-2018 03:05 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you try writing it out to hdfs location where you have write permissions ?
Master Collaborator
Created ‎11-21-2018 03:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
its work from pyspark shell but no from jupyter with pyspark kernel.
