Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error saveAsTextFile with pyspark

Highlighted

Error saveAsTextFile with pyspark

Super Collaborator

I have this error from pyspark, when I type this, any suggestion?

counts.saveAsTextFile("file:///tmp/JUPYTERR_out7")
Caused by: java.io.IOException: Failed to rename DeprecatedRawLocalFileStatus{path=file:/tmp/JUPYTERR_out7/_temporary/0/task_20181121142733_0009_m_000001/part-00001; isDirectory=false; length=6448; replication=1; blocksize=33554432; modification_time=1542806853000; access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false} to file:/tmp/JUPYTERR_out7/part-00001
2 REPLIES 2

Re: Error saveAsTextFile with pyspark

Contributor

Can you try writing it out to hdfs location where you have write permissions ?

Re: Error saveAsTextFile with pyspark

Super Collaborator

its work from pyspark shell but no from jupyter with pyspark kernel.