Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to submit spark jobs from a remote machine?

avatar

Steps to be performed before submiting

the spark jobs from a remote machine. I have tried adding the core.xml and yarn.xml file form master to the remote machine in the spark conf foldern. Still its not working. Can anyone suggest how to perform this step.

1 REPLY 1

avatar
@Deepak Nayak

We can submit the spark jobs on remote cluster using the livy server using Rest calls. Below are the couple of links for examples:

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-livy-rest-interface

https://github.com/cloudera/livy

Hope this helps you.