Oozie Spark-Action workflow taking more time to finish compared to spark-submit:
I'm trying to run Spark-SQL Scala jar , if I execute the jar using spark-submit it is taking 2 min for the same jar if i execute it through Oozie Spark -Action job is running for 12 min . I'm running the jar with same number of executors/ exec memory/ driver memory /driver cores / yarn mode in both spark-submit and Oozie spark-action. I even tried giving different memory settings while running Oozie Spark -Action (OOZIE_MAP_REDUCE_JAVA_OPTS ) but nothing worked. Do we need to give more memory for oozie while running Oozie SPARK-Action Job or are there any oozie -spark settings ? Thanks for the help!!
... View more
