Jars files using Yarn in Spark

[#1]

Hi,

I have a .jar file in one of the location (local or cluster) and I want to run that file using spark-shell in Yarn mode.. Can any one help me with this??

Thanks

[#2]

Try like this command.

spark-shell --jars /app/spark/a.jar,/app/spark/b.jar

[#3]

Thanks Peter Kim,

Correct me if I my understanding is wrong,

spark-shell --jars "location" of jar files --master "yarn"

is this command correct? Do I need to write anything else along with --master "yarn" this for Yarn

[#4]

If you're using hadoop cluster with ambari of hortonworks, then you don't have to use that --master yarn parameter. Cause' spark service mode of HDP cluster is installed to yarn mode basically.

[#5 · Accepted solution]

@Harshil Gala

spark-shell --jars <path-to-jar> --master yarn --deploy-mode client/cluster (you can choose client or accordingly)