Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

scala program to load data from rdbms to elasticsearch but showing error

Highlighted

scala program to load data from rdbms to elasticsearch but showing error

New Contributor

i have written scala program to load data from rdbms to elasticsearch but showing error like

error: value saveToEs is not a member of org.apache.spark.sql.DataFrame

here is my program...

import org.apache.spark.SparkContext

import org.elasticsearch.spark.rdd.EsSpark

spark.conf.set("es.index.auto.create", "true")

val elasticOptions = Map("es.mapping.id" -> "id", "es.nodes" -> "localhost", "es.port" -> "9200", "es.index.auto.create" -> "yes" )

val sqlContext = new org.apache.spark.sql.SQLContext(sc)

val mydata = sqlContext.load("jdbc", Map("url" ->"jdbc:postgresql://localhost:5432/testdb?user=postgres&password=postgres", "dbtable" -> "mytable"))

mydata.saveToEs(s"$elasticIndex/$elasticMappingLib", elasticOptions)

2 REPLIES 2
Highlighted

Re: scala program to load data from rdbms to elasticsearch but showing error

Cloudera Employee

Hi @Ranjit Chavan

could you please try adding:

import org.elasticsearch.spark._

Regards,

Samir

Re: scala program to load data from rdbms to elasticsearch but showing error

Cloudera Employee

Hi @Ranjit Chavan did you solve the issue?

Don't have an account?
Coming from Hortonworks? Activate your account here