Hi, I have been using oozie to schedule spark jobs. Trying to deploy a spark job in 2.x cluster using spark action available in Oozie.
In my job.properties, I have the following
<code> nameNode=hdfs://hostname:8020 jobTracker=hostname:8050 master=yarn-cluster queueName=default oozie.use.system.libpath=true
Whenever i submit the oozie job, i have been receiving this error
ErrorCode [JA009], Message [JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.] org.apache.oozie.action.ActionExecutorException: JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:457)
Do i need to change anything in oozie configuration file regarding the mapreduce frame work? Currently it has been set to "yarn"
Is it failing for this particular Spark job or all the Oozie jobs? As Mark pointed out, setting 'mapreduce.framework.name=yarn' should be fine. Also in case you have enabled Yarn RM HA, you should use jobTracker port 8032 instead of 8050.
Yes..It is failing for all the oozie jobs. I have changed the job tracker port to 8032 and tried again. But still i have been facing the same issue !