How to write rest api for spark job?

[#1]

Hi guys,

I have written spark job which read csv file and on top of csv data am calling dataframe api to extract the profile information.

I am manually submiting spark job by using spark-submit command but I want to write jersey rest api as wrapper on top of spark job and which is i have done but am not getting

How I should launch my spark job from rest service method?

I am giving call to rest api using tomcat,so is it possible to launch spark job in tomcat web application? If not then,

Is there any solution to achieve this functionality?

Thanks in advance.

[#2 · Accepted solution]

you can use spark job rest server which will allow you to submit the job using rest api.. please follow https://github.com/spark-jobserver/spark-jobserver

[#3]

Thanks Rajkumar,

Do we need some extra configuration to have spark job server?

If yes,then what are those steps?

Can we pass command line option to curl call while submitting spark job?

[#4]

you dont need any configuration params and you can submit spark job using curl commands