2016-11-01 09:20:33,497 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1474966402164_0109_000001
2016-11-01 09:20:33,646 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2016-11-01 09:20:33,805 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 109 cluster_timestamp: 1474966402164 } attemptId: 1 } keyId: -186141110)
2016-11-01 09:20:33,806 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: RM_DELEGATION_TOKEN, Service: 192.168.0.130:8032, Ident: (owner=hadoop, renewer=oozie mr token, realUser=hadoop, issueDate=1477963223337, maxDate=1478568023337, sequenceNumber=96, masterKeyId=36)
2016-11-01 09:20:33,819 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2016-11-01 09:20:33,820 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2016-11-01 09:20:34,311 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2016-11-01 09:20:34,314 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2016-11-01 09:20:34,314 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2016-11-01 09:20:34,315 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2016-11-01 09:20:34,315 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2016-11-01 09:20:34,316 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2016-11-01 09:20:34,316 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2016-11-01 09:20:34,317 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2016-11-01 09:20:34,343 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://dev0.youedata.com:9000]
2016-11-01 09:20:34,358 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://dev0.youedata.com:9000]
2016-11-01 09:20:34,371 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://dev0.youedata.com:9000]
2016-11-01 09:20:34,380 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2016-11-01 09:20:34,406 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2016-11-01 09:20:34,460 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2016-11-01 09:20:34,472 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSinkAdapter: Sink ganglia started
2016-11-01 09:20:34,510 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2016-11-01 09:20:34,510 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2016-11-01 09:20:34,517 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1474966402164_0109 to jobTokenSecretManager
2016-11-01 09:20:34,590 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1474966402164_0109 because: not enabled; too much RAM;
2016-11-01 09:20:34,605 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1474966402164_0109 = 0. Number of splits = 1
2016-11-01 09:20:34,605 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1474966402164_0109 = 0
2016-11-01 09:20:34,605 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1474966402164_0109Job Transitioned from NEW to INITED
2016-11-01 09:20:34,606 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1474966402164_0109.
2016-11-01 09:20:34,623 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2016-11-01 09:20:34,630 INFO [Socket Reader #1 for port 46032] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 46032
2016-11-01 09:20:34,642 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2016-11-01 09:20:34,643 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-11-01 09:20:34,643 INFO [IPC Server listener on 46032] org.apache.hadoop.ipc.Server: IPC Server listener on 46032: starting
2016-11-01 09:20:34,643 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at dev3.youedata.com/192.168.0.133:46032
2016-11-01 09:20:34,694 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2016-11-01 09:20:34,699 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2016-11-01 09:20:34,702 WARN [main] org.apache.hadoop.http.HttpRequestLog: Jetty request log can only be enabled using Log4j
2016-11-01 09:20:34,706 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2016-11-01 09:20:34,709 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2016-11-01 09:20:34,709 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2016-11-01 09:20:34,711 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2016-11-01 09:20:34,711 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2016-11-01 09:20:34,912 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2016-11-01 09:20:34,913 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 44888
2016-11-01 09:20:34,913 INFO [main] org.mortbay.log: jetty-6.1.26
2016-11-01 09:20:34,932 INFO [main] org.mortbay.log: Extract jar:file:/youe/dmp-v0.1/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-common-2.7.3.jar!/webapps/mapreduce to ./tmp/Jetty_0_0_0_0_44888_mapreduce____.b395vj/webapp
2016-11-01 09:20:35,836 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:44888
2016-11-01 09:20:35,837 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 44888
2016-11-01 09:20:35,839 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2016-11-01 09:20:35,839 INFO [Socket Reader #1 for port 58994] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 58994
2016-11-01 09:20:35,842 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-11-01 09:20:35,842 INFO [IPC Server listener on 58994] org.apache.hadoop.ipc.Server: IPC Server listener on 58994: starting
2016-11-01 09:20:35,865 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2016-11-01 09:20:35,865 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2016-11-01 09:20:35,865 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2016-11-01 09:20:35,895 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at dev0.youedata.com/192.168.0.130:8030
2016-11-01 09:20:35,936 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: <memory:24500, vCores:32>
2016-11-01 09:20:35,936 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: default
2016-11-01 09:20:35,939 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2016-11-01 09:20:35,939 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10
2016-11-01 09:20:35,940 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
2016-11-01 09:20:35,944 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1474966402164_0109Job Transitioned from INITED to SETUP
2016-11-01 09:20:35,945 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2016-11-01 09:20:35,947 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2016-11-01 09:20:35,960 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1474966402164_0109Job Transitioned from SETUP to RUNNING
2016-11-01 09:20:35,979 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1474966402164_0109_m_000000 Task Transitioned from NEW to SCHEDULED
2016-11-01 09:20:35,980 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1474966402164_0109_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2016-11-01 09:20:35,981 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:<memory:3070, vCores:1>
2016-11-01 09:20:36,003 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1474966402164_0109, File: hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/hadoop/.staging/job_1474966402164_0109/job_1474966402164_0109_1.jhist
2016-11-01 09:20:36,938 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2016-11-01 09:20:36,960 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1474966402164_0109: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:116736, vCores:37> knownNMs=5
2016-11-01 09:20:37,967 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2016-11-01 09:20:37,968 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved dev4.youedata.com to /default-rack
2016-11-01 09:20:37,969 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1474966402164_0109_01_000002 to attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:37,970 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
2016-11-01 09:20:38,001 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved dev4.youedata.com to /default-rack
2016-11-01 09:20:38,001 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Job jar is not present. Not adding any jar to the list of resources.
2016-11-01 09:20:38,012 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /tmp/hadoop-yarn/staging/hadoop/.staging/job_1474966402164_0109/job.xml
2016-11-01 09:20:38,208 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #1 tokens and #1 secret keys for NM use for launching container
2016-11-01 09:20:38,208 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 2
2016-11-01 09:20:38,208 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
2016-11-01 09:20:38,772 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1474966402164_0109_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2016-11-01 09:20:38,776 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1474966402164_0109_01_000002 taskAttempt attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:38,777 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:38,778 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : dev4.youedata.com:56717
2016-11-01 09:20:38,821 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1474966402164_0109_m_000000_0 : 13562
2016-11-01 09:20:38,822 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1474966402164_0109_m_000000_0] using containerId: [container_1474966402164_0109_01_000002 on NM: [dev4.youedata.com:56717]
2016-11-01 09:20:38,824 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1474966402164_0109_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2016-11-01 09:20:38,824 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1474966402164_0109_m_000000 Task Transitioned from SCHEDULED to RUNNING
2016-11-01 09:20:38,971 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1474966402164_0109: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:112640, vCores:36> knownNMs=5
2016-11-01 09:20:40,339 INFO [Socket Reader #1 for port 58994] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1474966402164_0109 (auth:SIMPLE)
2016-11-01 09:20:40,353 INFO [IPC Server handler 0 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1474966402164_0109_m_000002 asked for a task
2016-11-01 09:20:40,353 INFO [IPC Server handler 0 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1474966402164_0109_m_000002 given task: attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:46,671 INFO [IPC Server handler 3 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1474966402164_0109_m_000000_0 is : 0.0
2016-11-01 09:20:47,038 INFO [IPC Server handler 3 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1474966402164_0109_m_000000_0 is : 1.0
2016-11-01 09:20:47,461 INFO [IPC Server handler 4 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:47,462 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1474966402164_0109_m_000000_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2016-11-01 09:20:47,462 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1474966402164_0109_m_000000_0 given a go for committing the task output.
2016-11-01 09:20:47,463 INFO [IPC Server handler 6 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:47,463 INFO [IPC Server handler 6 on 58994] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1474966402164_0109_m_000000_0:true
2016-11-01 09:20:47,779 INFO [IPC Server handler 1 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1474966402164_0109_m_000000_0 is : 1.0
2016-11-01 09:20:47,782 INFO [IPC Server handler 2 on 58994] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:47,785 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1474966402164_0109_m_000000_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_CONTAINER_CLEANUP
2016-11-01 09:20:47,786 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_1474966402164_0109_01_000002 taskAttempt attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:47,786 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:47,786 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : dev4.youedata.com:56717
2016-11-01 09:20:47,798 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1474966402164_0109_m_000000_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED
2016-11-01 09:20:47,804 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1474966402164_0109_m_000000_0
2016-11-01 09:20:47,804 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1474966402164_0109_m_000000 Task Transitioned from RUNNING to SUCCEEDED
2016-11-01 09:20:47,807 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
2016-11-01 09:20:47,808 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1474966402164_0109Job Transitioned from RUNNING to COMMITTING
2016-11-01 09:20:47,808 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT
2016-11-01 09:20:47,983 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
2016-11-01 09:20:47,987 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1474966402164_0109_01_000002
2016-11-01 09:20:47,988 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
2016-11-01 09:20:47,988 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1474966402164_0109_m_000000_0: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

2016-11-01 09:20:49,775 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent 
2016-11-01 09:20:49,775 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1474966402164_0109Job Transitioned from COMMITTING to SUCCEEDED
2016-11-01 09:20:49,776 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry
2016-11-01 09:20:49,776 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true
2016-11-01 09:20:49,776 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified that shouldUnregistered is: true
2016-11-01 09:20:49,776 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2016-11-01 09:20:49,776 INFO [Thread-69] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true
2016-11-01 09:20:49,776 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services
2016-11-01 09:20:49,776 INFO [Thread-69] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
2016-11-01 09:20:51,717 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/hadoop/.staging/job_1474966402164_0109/job_1474966402164_0109_1.jhist to hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109-1477963232367-hadoop-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell%2Dspark%3AA%3Dshell%2Dnode%3A-1477963249774-1-0-SUCCEEDED-default-1477963235941.jhist_tmp
2016-11-01 09:20:52,964 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109-1477963232367-hadoop-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell%2Dspark%3AA%3Dshell%2Dnode%3A-1477963249774-1-0-SUCCEEDED-default-1477963235941.jhist_tmp
2016-11-01 09:20:53,162 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/hadoop/.staging/job_1474966402164_0109/job_1474966402164_0109_1_conf.xml to hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109_conf.xml_tmp
2016-11-01 09:20:53,614 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109_conf.xml_tmp
2016-11-01 09:20:54,353 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109.summary_tmp to hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109.summary
2016-11-01 09:20:55,252 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109_conf.xml_tmp to hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109_conf.xml
2016-11-01 09:20:55,276 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109-1477963232367-hadoop-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell%2Dspark%3AA%3Dshell%2Dnode%3A-1477963249774-1-0-SUCCEEDED-default-1477963235941.jhist_tmp to hdfs://dev0.youedata.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1474966402164_0109-1477963232367-hadoop-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell%2Dspark%3AA%3Dshell%2Dnode%3A-1477963249774-1-0-SUCCEEDED-default-1477963235941.jhist
2016-11-01 09:20:55,277 INFO [Thread-69] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2016-11-01 09:20:55,278 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to 
2016-11-01 09:20:55,279 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://dev0.youedata.com:19888/jobhistory/job/job_1474966402164_0109
2016-11-01 09:20:55,283 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.
2016-11-01 09:20:56,284 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
2016-11-01 09:20:56,285 INFO [Thread-69] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://dev0:9000 /tmp/hadoop-yarn/staging/hadoop/.staging/job_1474966402164_0109
2016-11-01 09:20:56,647 INFO [Thread-69] org.apache.hadoop.ipc.Server: Stopping server on 58994
2016-11-01 09:20:56,647 INFO [IPC Server listener on 58994] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 58994
2016-11-01 09:20:56,648 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
2016-11-01 09:20:56,648 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder