Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Type Error when attempting Linear Regression

Solved Go to solution

Type Error when attempting Linear Regression

Guru
import org.apache.spark.mllib.regression.LinearRegressionWithSGD
import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.ml.feature.{OneHotEncoder, StringIndexer}
import sqlContext.implicits._


val df = sqlContext.sql("select mnemonic, average, median, stddev from wellbook.curve_statistics")


val indexer = new StringIndexer()
  .setInputCol("mnemonic")
  .setOutputCol("mnemonicIndex")
  .fit(df)
val indexed = indexer.transform(df)


val encoder = new OneHotEncoder().setInputCol("mnemonicIndex").
  setOutputCol("mnemonicVec")
val encoded = encoder.transform(indexed)
val data = encoded.select("mnemonicVec", "average", "median", "stddev")


val parsedData = data.map(row => LabeledPoint(row.getDouble(0), row.getAs[Vector](1)))

<console>:297: error: kinds of the type arguments (Vector) do not conform to the expected kinds of the type parameters (type T). Vector's type parameters do not match type T's expected parameters: type Vector has one type parameter, but type T has none val parsedData = data.map(row => LabeledPoint(row.getDouble(0), row.getAs[Vector](1))

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Type Error when attempting Linear Regression

New Contributor

In addition to Vectors, you need to import the Spark Vector class explicitly since Scala imports its in-built Vector type by default. Try this:

import org.apache.spark.mllib.linalg.{Vector, Vectors}
6 REPLIES 6

Re: Type Error when attempting Linear Regression

New Contributor

Which version of Spark and HDP are you using?

Re: Type Error when attempting Linear Regression

Guru
@Dhruv Kumar

Spark1.4.1 and HDP2.3.2

Re: Type Error when attempting Linear Regression

Expert Contributor

Vedant, give this a shot:

val parsedData = data.map(row => LabeledPoint(row.getDouble(0), row.asInstanceOf[Vector](1)))

Re: Type Error when attempting Linear Regression

Guru

@Joe Widen I tried it earlier and gave me the same error.

Highlighted

Re: Type Error when attempting Linear Regression

New Contributor

In addition to Vectors, you need to import the Spark Vector class explicitly since Scala imports its in-built Vector type by default. Try this:

import org.apache.spark.mllib.linalg.{Vector, Vectors}

Re: Type Error when attempting Linear Regression

Guru

@Dhruv Kumar Thanks it worked.