Reply
New Contributor
Posts: 1
Registered: ‎04-25-2018

Unable to save Spark DataFrame to Solr

My Environment:

solr-spec - 4.10.3

solr-impl - 4.10.3-cdh5.13.0

Spark 1.6

spark-solr-3.0.2

spark-solr-3.0.2-shaded

solr-solrj-6.5.1

 

While I am trying to save the Spark Dataframe 'df.write().format("solr").options(options).save()' I am getting the following error:

Exception in thread "main" java.lang.AbstractMethodError: solr.DefaultSource.createRelation(Lorg/apache/spark/sql/SQLContext;Lorg/apache/spark/sql/SaveMode;Lscala/collection/immutable/Map;Lorg/apache/spark/sql/DataFrame;)Lorg/apache/spark/sql/sources/BaseRelation;
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:222)
at org.apache.spark.sql.DataFrameWriter.dataSource$lzycompute$1(DataFrameWriter.scala:181)
at org.apache.spark.sql.DataFrameWriter.org$apache$spark$sql$DataFrameWriter$$dataSource$1(DataFrameWriter.scala:181)
at org.apache.spark.sql.DataFrameWriter$$anonfun$save$1.apply$mcV$sp(DataFrameWriter.scala:188)
at org.apache.spark.sql.DataFrameWriter.executeAndCallQEListener(DataFrameWriter.scala:154)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:188)
at Test.main(Test.java:65)

 

New Contributor
Posts: 1
Registered: ‎06-05-2018

Re: Unable to save Spark DataFrame to Solr

Unfortunately, the version of Solr in Cloudera 5.x does not support that. You can write a Spark app to index a batch of  org.apache.solr.common.SolrInputDocument objects into  org.apache.solr.client.solrj.impl.CloudSolrServer.

New Contributor
Posts: 4
Registered: ‎10-13-2017

Re: Unable to save Spark DataFrame to Solr

Would it be possible in CDH6 with 7-th version of Solr?

Announcements