Spark JavaRDD savaasTextFile

[#1]

Hi,

I have JavaRDD, i want to save that into hdfs. am using fallowing code:

event1.saveAsTextFile("");

its creating folfer in hdfs but not writing anything into hdfs system and its not throwing any error too.

[#2]

If you use Spark Streaming and the event RDD is empty then Spark will write a folder and not add any content.

see e.g. Deleting Directory in HDFS using Spark for how to avoid this.

If it is not Streaming, verify the JavaRDD is not empty

[#3]

@Bernhard Walter Its streaming only. but before writing contents into hdfs ,I printed out contents. its displaying content too

[#4]

I was able to solve the problem using coalease function.

[#5]

@vnandigam , are you running literally this code?

event1.saveAsTextFile("");

Because, the saveAsTextFile expects one argument to be passed, a path to the file, if you pass an empty string as path, you should not expect your rdd data to be saved into any file.

http://spark.apache.org/docs/latest/api/java/org/apache/spark/api/java/JavaRDD.html#saveAsTextFile(j...

[#6]

@pbarna Hi, I removed path of my file here. I already passed destination follder