Created 03-21-2017 10:36 AM
I have JavaRDD, i want to save that into hdfs. am using fallowing code:
its creating folfer in hdfs but not writing anything into hdfs system and its not throwing any error too.
Created 03-21-2017 11:25 AM
If you use Spark Streaming and the event RDD is empty then Spark will write a folder and not add any content.
see e.g. Deleting Directory in HDFS using Spark for how to avoid this.
If it is not Streaming, verify the JavaRDD is not empty
Created 03-21-2017 11:35 AM
@Bernhard Walter Its streaming only. but before writing contents into hdfs ,I printed out contents. its displaying content too
Created 03-22-2017 04:34 AM
I was able to solve the problem using coalease function.
Created 03-21-2017 01:46 PM
@vnandigam , are you running literally this code?
Because, the saveAsTextFile expects one argument to be passed, a path to the file, if you pass an empty string as path, you should not expect your rdd data to be saved into any file.
Created 03-22-2017 04:33 AM