Type Error when attempting Linear Regression

[#1]

import org.apache.spark.mllib.regression.LinearRegressionWithSGD
import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.ml.feature.{OneHotEncoder, StringIndexer}
import sqlContext.implicits._


val df = sqlContext.sql("select mnemonic, average, median, stddev from wellbook.curve_statistics")


val indexer = new StringIndexer()
  .setInputCol("mnemonic")
  .setOutputCol("mnemonicIndex")
  .fit(df)
val indexed = indexer.transform(df)


val encoder = new OneHotEncoder().setInputCol("mnemonicIndex").
  setOutputCol("mnemonicVec")
val encoded = encoder.transform(indexed)
val data = encoded.select("mnemonicVec", "average", "median", "stddev")


val parsedData = data.map(row => LabeledPoint(row.getDouble(0), row.getAs[Vector](1)))

<console>:297: error: kinds of the type arguments (Vector) do not conform to the expected kinds of the type parameters (type T). Vector's type parameters do not match type T's expected parameters: type Vector has one type parameter, but type T has none val parsedData = data.map(row => LabeledPoint(row.getDouble(0), row.getAs[Vector](1))

[#2]

Which version of Spark and HDP are you using?

[#3]

@Dhruv Kumar

Spark1.4.1 and HDP2.3.2

[#4]

Vedant, give this a shot:

val parsedData = data.map(row => LabeledPoint(row.getDouble(0), row.asInstanceOf[Vector](1)))

[#5]

@Joe Widen I tried it earlier and gave me the same error.

[#6 · Accepted solution]

In addition to Vectors, you need to import the Spark Vector class explicitly since Scala imports its in-built Vector type by default. Try this:

import org.apache.spark.mllib.linalg.{Vector, Vectors}

[#7]

@Dhruv Kumar Thanks it worked.