In the logs, I get this classNotFoundException .NodeManager fails in a couple of seconds when started. Rest of the YARN components are working fine except for Nodemanager. YARN service check times out and never completes. YARN logs: 2016-05-05 12:44:24,449 INFO event.AsyncDispatcher (AsyncDispatcher.java:register(208)) - Registering 2016-05-05 12:44:24,733 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config. 2016-05-05 12:44:24,734 INFO containermanager.AuxServices (AuxServices.java:addService(72)) - Adding auxiliary service httpshuffle, "mapreduce_shuffle" 2016-05-05 12:44:24,834 FATAL containermanager.AuxServices (AuxServices.java:serviceInit(145)) - Failed to initialize spark_shuffle java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.spark.network.yarn.YarnShuffleService not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227) at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:121) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:291) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:537) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:585) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.spark.network.yarn.YarnShuffleService not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219) ... 10 more Caused by: java.lang.ClassNotFoundException: Class org.apache.spark.network.yarn.YarnShuffleService not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193) ... 11 more You can check the complete logs here: http://warehouse.swtched.com:8088/logs/yarn-yarn-nodemanager-warehouse.swtched.com.log ... View more

What exactly is xmls attribute in oozie workflows? I am trying to run a sqoop command but I keep getting ava.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found Now, I have tried everything suggested. Set the permissions right. Have /share/lib in /user/oozie. Running oozie jobs from oozie user only. Just wondering if xmlns could be the problem? I am using ambari. This is what I get when I check my versions: sqoop version Warning: /usr/hdp/2.4.0.0-169/hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /usr/hdp/2.4.0.0-169/accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 16/05/04 12:54:07 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.0.0-169 Sqoop 1.4.6.2.4.0.0-169 git commit id c327581e17e308322d77a2ae245d0287c0e488d4 Compiled by jenkins on Wed Feb 10 08:35:28 UTC 2016 [oozie@warehouse root]$ oozie version Oozie client build version: 4.2.0.2.4.0.0-169 My workflow has; <workflow-app name="once-a-day" xmlns="uri:oozie:workflow:0.1"> <start to="sqoopAction"/> <action name="sqoopAction"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <command>list-tables --connect jdbc:mysql://FQDN/erp --username hive --password hive </command> <archive>/lib/mysql-connector-java-5.1.38-bin.jarz#mysql-connector-java-5.1.38-bin.jar</archive> </sqoop> <ok to="end"/> <error to="killJob"/> </action> <kill name="killJob"> <message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message> </kill> <end name="end" /> </workflow-app> job.props: nameNode=hdfs://FQDN:8020 jobTracker=FQDN:8050 queueName=default oozie.libpath=/user/oozie/share/lib/, /user/oozie/share/lib/lib_20160427195409 oozie.use.system.libpath=true oozie.wf.application.path=${nameNode}/user/${user.name}/scheduledimport start=2016-04-26T00:00Z end=2016-12-31T00:00Z workflowAppUri=${nameNode}/user/${user.name}/scheduledimport ... View more