Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to create jar file from spark scala file?

avatar
Contributor
 
4 REPLIES 4

avatar
Expert Contributor

I think that the best option for compiling scala Spark code is to use sbt,which is a tool for managing dependencies. You can do the same with Maven anyway, as you prefer.

avatar

See http://spark.apache.org/docs/latest/quick-start.html#self-contained-applications "Self-Contained Applications" for all languages

avatar
Super Guru

@AKILA VEL

Though there are many ways to do that but you can use sbt tool to build your application jar, below is a good example doc to build a jar and run it on spark

https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-first-app.html

avatar

In case you are looking for a Maven project to build Spark/Scala. Here is an example https://github.com/vinayshukla/SparkDemo1

Note it was for Spark 1.1.0 but you can change the version.