I am using the spark to train model and save it on file system as using org.apache.spark.ml.PipelineModel.save() .
This works fine when I am running the spark in local mode but when I am running the spark in standalone mode where 2 workers are there , it is training the model correctly but when saving on the file it is storing the partial result .And the file system I am using is Local one i.e file:// .
Could someone please point out what is the issue here.
And is it an issue to store the trained PipelineModel on local file system.
Same code works when I use HDFS file system.
Spark version I am using is : 2.0.0
Thanks in Advance ,