I am trying to run spark worklow in oozie. I have created workflow using Hue with Spark Action and given the jar, class details properly. It has been failing continously and really making frustrated. I tried all the hell of options with no use and i could not resolve in for a month now.
master: tried spark standalone master, yarn, local[*] and local, nothing worked out.
Question 1: For each and every spark job, it submits a MAPREDUCE and SPARK job. The job gets accepted and then both MR and Spark jobs shows up in Job Browser and after 20 to 25 minutes, they will fail and no logs will appear to debug. What should i do, where should i search and it has been really frustrating experience.
I have tried with all the masters and all the options suggested over the internet.