Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Livy rest api with web app

avatar

Hi,

I have a requirement to train a predictive machine learning model from web application. User can change tuning parameters/ algorithm from User Interface which goes to Spark job and returns regression metric.

Right now, I have developed a web application REST API with Jersey and Jetty.

I have a Spark machine learning application which implements Livy job. I am referring this jar from web application.

The flow is as follows. User changes tuning parameter/ algorithm from UI -> Jersey API calls Livy client with parameters -> Livy job runs in YARN and returns metric results -> Jersey returns metric results to UI.

Am I doing it correct? Is there any other way that I can interact with SPARK from web app? I am creating LivyClient singleton object and I am using this object for submitting job throughout the application. The code was working fine. I am getting the below exception now. Do I have to create LivyClient object and upload jar (Spark machine learning application with Livy) each and every time?

javax.servlet.ServletException: java.util.concurrent.ExecutionException: java.io.IOException: Not Found: "Session '339' not found."

1 ACCEPTED SOLUTION

avatar
Super Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
2 REPLIES 2

avatar
Super Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar

@Bikas, Is this correct to create LivyClient object during application start up and use that object for all the API requests to submit jar?