Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to run Spark job from Oozie Workflow on HDP/hue

avatar
Expert Contributor

I have created a small java program for Spark. It works with "spark-submit" command. I like to run it from Oozie workflow. It seems HDP 2.3 has a capability to run Spark job from Oozie workflow, but on Hue's GUI, I don't have a choice of Spark job to include into a workflow. How do I do?

1 ACCEPTED SOLUTION

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
13 REPLIES 13

avatar
Explorer

For your review @Artem Ervits

https://na9.salesforce.com/articles/en_US/How_To/How-to-run-Spark-Action-in-oozie-of-HDP-2-3-0?popup...

While I was not able to see the contents of this link directly (had to have somebody extract for me), perhaps you can as a Hortonworks insider.

avatar
Master Mentor

awesome! I will try this out and maybe publish an article @cmuchinsky

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Expert Contributor

I also tested HiveContext so that Hive processing works in Spark memory. It works.