08-24-2017 08:58 PM
I am trying to run the Spark action in Oozie with spark .jar
Provided below values in Hue Oozie Spark action,
Jar/py Name: solution.jar
Main Class: Module.final_solution
Below are the properties:
Spark Master: yarn-master
App Name: Final Solution
Getting below error while running Oozie action with above configurations,
Same jar working fine from Edge node with Spark submit.
Any help appriciated.
08-28-2017 05:17 AM
This looks like Hue might be having problems communicating with Yarn. Have you tried submitting the job.properties from the Edge node using
oozie job -submit
Is hue able to submit a hello-world oozie job and monitor it? Are you able to submit other jobs via Hue (Hive/pig, etc)?
09-01-2017 06:33 AM
Yes, other jobs are wokring fine.
I am on CDH 5.9, so just tried with jar path in options list which is atleast running the job now.
I am encountering below error while running the job,
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, requirement failed java.lang.IllegalArgumentException: requirement failed