I have HDP 2.4 installed on CentOS 6.8 on 6 virtual KVM instances based on two physical machines. I have been having problems with oozie jobs where the work flows call either hive or spark based actions. The error that I have encountered is 2016-08-18 11:01:18,134 WARN ActionStartXCommand:523 - SERVER[hc1m1.nec.co.nz] USER[oozie] GROUP[-] TOKEN[] APP[PoleLocationsForNec] JOB[0000001-160818105046419-oozie-oozi-W] ACTION[0000001-160818105046419-oozie-oozi-W@hive-select-data] Error starting action [hive-select-data]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.] org.apache.oozie.action.ActionExecutorException: JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. I have encountered this error on jobs that did already work so I cant think what has changed. My work flow looks like this <workflow-app name='PoleLocationsForNec' xmlns="uri:oozie:workflow:0.5" xmlns:sla="uri:oozie:sla:0.2"> <start to='hive-select-data'/> <action name="hive-select-data"> <hive xmlns="uri:oozie:hive-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <script>select_pole_locations_for_nec.hql</script> </hive> <ok to="hdfs-move-file"/> <error to="fail"/> </action> <action name="hdfs-move-file"> <fs> <move source='${sourceFile}' target='${targetFile}${SaveDateString}'/> </fs> <ok to="sqoop-copy"/> <error to="fail"/> </action> <action name="sqoop-copy"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <arg>export</arg> <arg>--options-file</arg> <arg>${sqoopOptFile}</arg> <file>${sqoopOptFile}</file> </sqoop> <ok to="cleanup"/> <error to="fail"/> </action> <action name="cleanup"> <fs> <delete path='${triggerFileDir}' /> </fs> <ok to="end"/> <error to="fail"/> </action> <kill name='fail'> <message> An error occurred - message[${wf:errorMessage(wf:lastErrorNode())}] </message> </kill> <end name='end'/> </workflow-app> and my job configuration looks like this. <configuration> <property> <name>hdfsUser</name> <value>oozie</value> </property> <property> <name>WaitForThisInputData</name> <value>hdfs://hc1m1.nec.co.nz:8020/mule/sheets/input/PoleLocationsForNec/trigger/</value> </property> <property> <name>wfPath</name> <value>hdfs://hc1m1.nec.co.nz:8020/user/oozie/wf/PoleLocationsForNec/</value> </property> <property> <name>user.name</name> <value>oozie</value> </property> <property> <name>sqoopOptFile</name> <value>sqoop_options_file.opt</value> </property> <property> <name>oozie.coord.application.path</name> <value>hdfs://hc1m1.nec.co.nz:8020/user/oozie/wf/PoleLocationsForNec</value> </property> <property> <name>sourceFile</name> <value>hdfs://hc1m1.nec.co.nz:8020/mule/sheets/input/PoleLocationsForNec/Pole_locations_for_NEC_edit.csv</value> </property> <property> <name>mapreduce.job.user.name</name> <value>oozie</value> </property> <property> <name>Execution</name> <value>FIFO</value> </property> <property> <name>triggerFileDir</name> <value>hdfs://hc1m1.nec.co.nz:8020/mule/sheets/input/PoleLocationsForNec/trigger/</value> </property> <property> <name>coordFreq</name> <value>30</value> </property> <property> <name>Concurrency</name> <value>1</value> </property> <property> <name>targetFile</name> <value>/mule/sheets/store/Pole_locations_for_NEC_edit.csv</value> </property> <property> <name>jobTracker</name> <value>hc1r1m2.nec.co.nz:8050</value> </property> <property> <name>startTime</name> <value>2016-07-31T12:01Z</value> </property> <property> <name>wfProject</name> <value>PoleLocationsForNec</value> </property> <property> <name>targetDir</name> <value>/mule/sheets/store/</value> </property> <property> <name>dataFreq</name> <value>30</value> </property> <property> <name>nameNode</name> <value>hdfs://hc1m1.nec.co.nz:8020</value> </property> <property> <name>doneFlag</name> <value>done_flag.dat</value> </property> <property> <name>oozie.libpath</name> <value>hdfs://hc1m1.nec.co.nz:8020/user/oozie/share/lib</value> </property> <property> <name>oozie.use.system.libpath</name> <value>true</value> </property> <property> <name>oozie.wf.rerun.failnodes</name> <value>true</value> </property> <property> <name>moveFile</name> <value>Pole_locations_for_NEC_edit.csv</value> </property> <property> <name>SaveDateString</name> <value>-20160817-230100</value> </property> <property> <name>triggerDir</name> <value>trigger/</value> </property> <property> <name>sourceDir</name> <value>hdfs://hc1m1.nec.co.nz:8020/mule/sheets/input/PoleLocationsForNec/</value> </property> <property> <name>oozie.wf.application.path</name> <value>hdfs://hc1m1.nec.co.nz:8020/user/oozie/wf/PoleLocationsForNec</value> </property> <property> <name>endTime</name> <value>2099-01-01T12:00Z</value> </property> <property> <name>TimeOutMins</name> <value>10</value> </property> <property> <name>timeZoneDef</name> <value>GMT+12:00</value> </property> <property> <name>workflowAppPath</name> <value>hdfs://hc1m1.nec.co.nz:8020/user/oozie/wf/PoleLocationsForNec</value> </property> </configuration> I have obviously researched this issue and understand that it related to the definition of mapreduce.framework.name or the hdfs / resource manager server address. But given that this job has worked in the past I thought that this error might be masking another issue. ?? The value of mapreduce.framework.name is defined in the following files /etc/hadoop/conf/mapred-site.xml => yarn-tez /etc/hive/conf/mapred-site.xml => yarn-tez /etc/oozie/conf/hadoop-config.xml => yarn /etc/pig/conf/pig-env.sh => yarn I have checked all of the logs but all I see is the JA009 error in the oozie logs. I just wondering whether anyone else has encountered this error or can suggest some other area that I can examine. ... View more