Using textFileStream in spark streaming , we are getting the new files from a directory but if any existing file loaded again with content modification in the directory . facing inconsistency like some times those are able to read in spark stream program some times not . what will be the reason and the solution for this .
Thanks in Advance .
Spark streaming is reading only new files. You should not change existing files, otherwise Spark Streaming is expected not to work properly.