Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to submit spark jobs from a remote machine?
Labels:
- Labels:
-
Apache Spark
Explorer
Created 08-17-2017 02:51 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Steps to be performed before submiting
the spark jobs from a remote machine. I have tried adding the core.xml and yarn.xml file form master to the remote machine in the spark conf foldern. Still its not working. Can anyone suggest how to perform this step.
1 REPLY 1
Guru
Created 08-17-2017 03:10 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Deepak Nayak
We can submit the spark jobs on remote cluster using the livy server using Rest calls. Below are the couple of links for examples:
https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-livy-rest-interface
https://github.com/cloudera/livy
Hope this helps you.
