Reply
Explorer
Posts: 12
Registered: ‎08-20-2015

Yarn RM REST api to submit map-reduce job

Hi - I am looking for some information about can we submit mapreduce job using Yarn RM REST Apis. I have a mapreduce jar which works well with hadoop jar commond unix shell. I want to submit the same job remotly via my web application. I am thinking to use Yarn RM REST Apis. Specially Cluster Applications API(Submit Application)

 

My Questions

1. Is this possible ? Is this the right approch.

2. Is there any sample code avaliable where I can see how to submit the MR job via REST Api. The documentation shows example for ShellCommond only which is not very clear to me.

 

Thanks in advance.

MG

 

Posts: 1,836
Kudos: 416
Solutions: 295
Registered: ‎07-31-2013

Re: Yarn RM REST api to submit map-reduce job

While it is possible to submit an application via the REST API of RM's
WebServices, its not a trivial thing to setup the same for a complex
application such as MapReduce2 (which will make use of every available API
hook).

You may find it easier instead to store your job on HDFS, and use an Oozie
workflow over it. Oozie supports a REST API that is far easier to use to
fire and control jobs:
http://archive.cloudera.com/cdh5/cdh/5/oozie/WebServicesAPI.html#Job_Submission
Explorer
Posts: 12
Registered: ‎08-20-2015

Re: Yarn RM REST api to submit map-reduce job

Thanks for the reply. We are moving on oozie wf webservice. I agree that is very easy as compare to yarn REST apis. Thanks.
Announcements