Hello, Can you please explain to me what kind of data I got when I use Spark Clustering from Mllib like the following KMeansModel clusters = KMeans.train(parsedData.rdd(), numClusters, numIterations); ... View more

I wrote Vector's (org.apache.spark.mllib.linalg.Vector) to the HDFS as the following public void writePointsToFile(Path path, FileSystem fs, Configuration conf, List<Vector> points) throws IOException { SequenceFile.Writer writer = SequenceFile.createWriter(conf, Writer.file(path), Writer.keyClass(LongWritable.class), Writer.valueClass(Vector.class)); long recNum = 0; for (Vector point : points) { writer.append(new LongWritable(recNum++), point); } writer.close(); } ( not sure that I used the right way to do that can't test it yet ) now I need to read this file as JavaRDD<Vector> because I want to use it in Spark Clustering K-mean but don't know how to do this. ... View more

I have a VectorWritable (org.apache.mahout.math.VectorWritable) which is coming from a sequence file generated by Mahout something like the following. publicvoid write(List<Vector> points,int clustersNumber,HdfsConnector connector)throwsIOException{this.writePointsToFile(newPath(connector.getPointsInput(),"pointsInput"), connector.getFs(), connector.getConf(), points);Path clusterCentroids =newPath(connector.getClustersInput(),"part-0");SequenceFile.Writer writer =SequenceFile.createWriter( connector.getConf(),Writer.file(clusterCentroids),Writer.keyClass(Text.class),Writer.valueClass(Kluster.class));List<Vector> centroids = getCentroids;for(int i =0; i < centroids.size(); i++){Vector vect = centroids.get(i);Kluster centroidCluster =newKluster(vect, i,newSquaredEuclideanDistanceMeasure()); writer.append(newText((centroidCluster).getIdentifier()), centroidCluster);} writer.close();} and I would like to convert that into Vector (org.apache.spark.mllib.linalg.Vectors) type Spark as JavaRDD<Vector> How can I do that in Java ? I've read something about sequenceFile in Spark but I couldn't figure out how to do it. ... View more