CDH-5.0.0 YARN repeat log "Ramping down all scheduled reduces:0"

[#1]

I'm use CDH-5.0.0 YRAN.  When i submit a java job use HUE-UI, the job status still show MAP 5%, REDUCE: 5%. 

But, if i kill task: "oozie:launcher.xxx", my job continue run, and the output  is ok.

Here is YARN logs:

Log Type: syslog

Log Length: 2428904

2014-05-13 16:00:35,925 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1399966076976_0005_000001
2014-05-13 16:00:36,135 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-05-13 16:00:36,144 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-05-13 16:00:36,227 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2014-05-13 16:00:36,227 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (org.apache.hadoop.yarn.security.AMRMTokenIdentifier@cf7e5b2)
2014-05-13 16:00:36,278 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: mapreduce.job, Service: 172.18.1.239:52141, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@2a55de3b)
2014-05-13 16:00:36,279 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: RM_DELEGATION_TOKEN, Service: 172.18.1.237:8032, Ident: (owner=hdfs, renewer=oozie mr token, realUser=oozie, issueDate=1399968023086, maxDate=1400572823086, sequenceNumber=10, masterKeyId=2)
2014-05-13 16:00:36,285 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: The specific max attempts: 2 for application: 5. Attempt num: 1 is last retry: false
2014-05-13 16:00:36,291 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2014-05-13 16:00:36,349 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-05-13 16:00:36,355 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-05-13 16:00:36,781 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2014-05-13 16:00:36,846 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2014-05-13 16:00:36,875 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2014-05-13 16:00:36,876 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2014-05-13 16:00:36,877 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2014-05-13 16:00:36,878 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2014-05-13 16:00:36,878 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2014-05-13 16:00:36,879 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2014-05-13 16:00:36,879 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2014-05-13 16:00:36,880 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2014-05-13 16:00:36,946 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2014-05-13 16:00:37,171 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2014-05-13 16:00:37,204 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2014-05-13 16:00:37,204 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2014-05-13 16:00:37,209 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1399966076976_0005 to jobTokenSecretManager
2014-05-13 16:00:37,285 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1399966076976_0005 because: not enabled; too much input;
2014-05-13 16:00:37,320 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1399966076976_0005 = 532305912. Number of splits = 4
2014-05-13 16:00:37,321 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1399966076976_0005 = 1
2014-05-13 16:00:37,321 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1399966076976_0005Job Transitioned from NEW to INITED
2014-05-13 16:00:37,322 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1399966076976_0005.
2014-05-13 16:00:37,343 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2014-05-13 16:00:37,348 INFO [Socket Reader #1 for port 57705] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 57705
2014-05-13 16:00:37,361 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2014-05-13 16:00:37,361 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2014-05-13 16:00:37,361 INFO [IPC Server listener on 57705] org.apache.hadoop.ipc.Server: IPC Server listener on 57705: starting
2014-05-13 16:00:37,362 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at cairo.hadoop/172.18.1.238:57705
2014-05-13 16:00:37,401 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2014-05-13 16:00:37,404 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2014-05-13 16:00:37,410 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2014-05-13 16:00:37,455 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2014-05-13 16:00:37,455 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2014-05-13 16:00:37,457 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2014-05-13 16:00:37,457 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2014-05-13 16:00:37,464 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 55186
2014-05-13 16:00:37,464 INFO [main] org.mortbay.log: jetty-6.1.26
2014-05-13 16:00:37,480 INFO [main] org.mortbay.log: Extract jar:file:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop-yarn/hadoop-yarn-common-2.3.0-cdh5.0.0.jar!/webapps/mapreduce to /tmp/Jetty_0_0_0_0_55186_mapreduce____.ekfg6/webapp
2014-05-13 16:00:37,627 INFO [main] org.mortbay.log: Started SelectChannelConnector@0.0.0.0:55186
2014-05-13 16:00:37,627 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app /mapreduce started at 55186
2014-05-13 16:00:37,994 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2014-05-13 16:00:37,997 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2014-05-13 16:00:37,997 INFO [Socket Reader #1 for port 56232] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 56232
2014-05-13 16:00:38,000 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2014-05-13 16:00:38,011 INFO [IPC Server listener on 56232] org.apache.hadoop.ipc.Server: IPC Server listener on 56232: starting
2014-05-13 16:00:38,026 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2014-05-13 16:00:38,026 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2014-05-13 16:00:38,026 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2014-05-13 16:00:38,056 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-05-13 16:00:38,059 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-05-13 16:00:38,098 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at miami.hadoop/172.18.1.237:8030
2014-05-13 16:00:38,175 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: maxContainerCapability: 1502
2014-05-13 16:00:38,176 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: root.hdfs
2014-05-13 16:00:38,179 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2014-05-13 16:00:38,194 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies : 500
2014-05-13 16:00:38,198 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1399966076976_0005Job Transitioned from INITED to SETUP
2014-05-13 16:00:38,199 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2014-05-13 16:00:38,273 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1399966076976_0005Job Transitioned from SETUP to RUNNING
2014-05-13 16:00:38,373 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved nantes.hadoop to /default
2014-05-13 16:00:38,385 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved cairo.hadoop to /default
2014-05-13 16:00:38,397 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved miami.hadoop to /default
2014-05-13 16:00:38,398 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1399966076976_0005_m_000000 Task Transitioned from NEW to SCHEDULED
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved nantes.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved cairo.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved miami.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1399966076976_0005_m_000001 Task Transitioned from NEW to SCHEDULED
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved nantes.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved cairo.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved miami.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1399966076976_0005_m_000002 Task Transitioned from NEW to SCHEDULED
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved nantes.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved cairo.hadoop to /default
2014-05-13 16:00:38,399 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved miami.hadoop to /default
2014-05-13 16:00:38,400 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1399966076976_0005_m_000003 Task Transitioned from NEW to SCHEDULED
2014-05-13 16:00:38,400 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1399966076976_0005_r_000000 Task Transitioned from NEW to SCHEDULED
2014-05-13 16:00:38,401 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1399966076976_0005_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-05-13 16:00:38,401 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1399966076976_0005_m_000001_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-05-13 16:00:38,401 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1399966076976_0005_m_000002_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-05-13 16:00:38,401 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1399966076976_0005_m_000003_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-05-13 16:00:38,401 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1399966076976_0005_r_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2014-05-13 16:00:38,402 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceReqt:1024
2014-05-13 16:00:38,405 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: reduceResourceReqt:1024
2014-05-13 16:00:38,418 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1399966076976_0005, File: hdfs://miami.hadoop:8020/user/hdfs/.staging/job_1399966076976_0005/job_1399966076976_0005_1.jhist
2014-05-13 16:00:39,178 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:4 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2014-05-13 16:00:39,208 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1399966076976_0005: ask=5 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:0, vCores:0> knownNMs=3
2014-05-13 16:00:39,208 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:39,208 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:39,209 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
2014-05-13 16:00:39,209 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 4
2014-05-13 16:00:40,212 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:40,212 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:40,212 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
2014-05-13 16:00:40,212 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 4
2014-05-13 16:00:41,214 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:41,214 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:41,214 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
2014-05-13 16:00:41,214 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 4
2014-05-13 16:00:42,218 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:42,218 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:42,218 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
2014-05-13 16:00:42,218 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 4
2014-05-13 16:00:43,222 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:43,222 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:43,222 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
2014-05-13 16:00:43,222 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 4
2014-05-13 16:00:44,226 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:44,226 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:44,226 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
2014-05-13 16:00:44,226 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 4
2014-05-13 16:00:45,230 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:45,230 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:45,230 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
2014-05-13 16:00:45,230 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 4
2014-05-13 16:00:46,232 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:0
2014-05-13 16:00:46,232 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 0
2014-05-13 16:00:46,232 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=0
... ...

[#2 · Accepted solution]

Please post your cluster's memory configuration, such as the resource MB offered by the NodeManagers, and individual MapReduce settings of AM, Map and Reduce task memories.

It appears that the cluster's unable to schedule more than 1 or 2 containers at a time, causing the job to eternally hang cause Oozie runs 2x AMs grabbing 2x containers already.