Support Questions
Find answers, ask questions, and share your expertise

how to load a CSV file in elasticsearch using pyspark?

please share some sample document to load a sample csv file to elasticsearch using pyspark

1 REPLY 1

@Rajesh AJ

Here is a link to the Elasticsearch for Apache Hadoop documentation: Elasticsearch for Apache Hadoop

The documentation does a very good job of showing examples for writing data to Elasticsearch using Spark. Most of the examples cover Scala or Java, but the end of that page does give an example using PySpark.

This blog may also be helpful: https://prasanthkothuri.wordpress.com/2016/06/17/integrating-hadoop-and-elasticsearch-part-2-queryin...

You may also find this blog article helpful for loading CSV files into Spark DataFrames: http://www.nodalpoint.com/spark-dataframes-from-csv-files/