Log Type: syslog
Log Upload Time: Sun Feb 14 23:44:17 -0500 2016
Log Length: 37661
2016-02-14 23:41:52,065 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1455503320604_0004_000001
2016-02-14 23:41:53,169 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-02-14 23:41:53,363 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2016-02-14 23:41:53,364 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 4 cluster_timestamp: 1455503320604 } attemptId: 1 } keyId: 1239204273)
2016-02-14 23:41:53,700 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:HDPCA, Ident: (HDFS_DELEGATION_TOKEN token 599 for ambari-qa)
2016-02-14 23:41:53,702 WARN [main] org.apache.hadoop.security.token.Token: Cannot find class for token kind kms-dt
2016-02-14 23:41:53,702 WARN [main] org.apache.hadoop.security.token.Token: Cannot find class for token kind kms-dt
Kind: kms-dt, Service: 192.168.0.102:9292, Ident: 00 0f 61 6d 62 61 72 69 2d 71 61 2d 48 44 50 43 41 02 72 6d 00 8a 01 52 e3 06 19 54 8a 01 53 07 12 9d 54 04 02
2016-02-14 23:41:53,702 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: TIMELINE_DELEGATION_TOKEN, Service: 192.168.122.40:8188, Ident: (owner=ambari-qa, renewer=yarn, realUser=, issueDate=1455507794615, maxDate=1456112594615, sequenceNumber=173, masterKeyId=272)
2016-02-14 23:41:53,741 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2016-02-14 23:41:53,752 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2016-02-14 23:41:54,107 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2016-02-14 23:41:54,108 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2016-02-14 23:41:55,898 WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2016-02-14 23:41:55,924 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2016-02-14 23:41:56,743 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2016-02-14 23:41:56,745 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2016-02-14 23:41:56,746 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2016-02-14 23:41:56,747 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2016-02-14 23:41:56,748 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2016-02-14 23:41:56,749 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2016-02-14 23:41:56,749 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2016-02-14 23:41:56,750 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2016-02-14 23:41:56,875 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore -  ignoring
2016-02-14 23:41:56,964 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore -  ignoring
2016-02-14 23:41:57,052 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore -  ignoring
2016-02-14 23:41:57,175 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2016-02-14 23:41:57,264 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2016-02-14 23:41:58,017 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-mrappmaster.properties,hadoop-metrics2.properties
2016-02-14 23:41:58,576 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2016-02-14 23:41:58,578 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2016-02-14 23:41:58,602 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1455503320604_0004 to jobTokenSecretManager
2016-02-14 23:41:59,592 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1455503320604_0004 because: not enabled; too much RAM;
2016-02-14 23:41:59,816 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1455503320604_0004 = 3550. Number of splits = 1
2016-02-14 23:41:59,817 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1455503320604_0004 = 1
2016-02-14 23:41:59,817 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1455503320604_0004Job Transitioned from NEW to INITED
2016-02-14 23:41:59,818 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1455503320604_0004.
2016-02-14 23:42:00,037 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2016-02-14 23:42:00,247 INFO [Socket Reader #1 for port 59305] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 59305
2016-02-14 23:42:01,090 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2016-02-14 23:42:01,954 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-02-14 23:42:02,525 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at Lnx1.localdomain.com/192.168.122.40:59305
2016-02-14 23:42:02,954 INFO [IPC Server listener on 59305] org.apache.hadoop.ipc.Server: IPC Server listener on 59305: starting
2016-02-14 23:42:10,022 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2016-02-14 23:42:10,577 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2016-02-14 23:42:10,777 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2016-02-14 23:42:10,931 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2016-02-14 23:42:11,017 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2016-02-14 23:42:11,018 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2016-02-14 23:42:11,132 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2016-02-14 23:42:11,132 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2016-02-14 23:42:36,641 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2016-02-14 23:42:36,682 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 40170
2016-02-14 23:42:36,682 INFO [main] org.mortbay.log: jetty-6.1.26.hwx
2016-02-14 23:42:37,001 INFO [main] org.mortbay.log: Extract jar:file:/hadoop/yarn/local/filecache/13/mapreduce.tar.gz/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.7.1.2.3.4.0-3485.jar!/webapps/mapreduce to /hadoop/yarn/local/usercache/ambari-qa/appcache/application_1455503320604_0004/container_e126_1455503320604_0004_01_000001/tmp/Jetty_0_0_0_0_40170_mapreduce____fyx0r/webapp
2016-02-14 23:42:46,734 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:40170
2016-02-14 23:42:46,734 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 40170
2016-02-14 23:42:46,898 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2016-02-14 23:42:46,919 INFO [Socket Reader #1 for port 41456] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 41456
2016-02-14 23:42:47,320 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-02-14 23:42:47,355 INFO [IPC Server listener on 41456] org.apache.hadoop.ipc.Server: IPC Server listener on 41456: starting
2016-02-14 23:42:47,793 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2016-02-14 23:42:47,793 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2016-02-14 23:42:47,793 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2016-02-14 23:42:48,071 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at Lnx1.localdomain.com/192.168.122.40:8030
2016-02-14 23:42:48,956 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: <memory:1024, vCores:2>
2016-02-14 23:42:48,956 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: default
2016-02-14 23:42:49,030 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2016-02-14 23:42:49,030 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10
2016-02-14 23:42:49,307 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
2016-02-14 23:42:49,447 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1455503320604_0004Job Transitioned from INITED to SETUP
2016-02-14 23:42:49,509 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2016-02-14 23:42:49,939 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1455503320604_0004Job Transitioned from SETUP to RUNNING
2016-02-14 23:42:50,385 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1455503320604_0004, File: hdfs://HDPCA:8020/user/ambari-qa/.staging/job_1455503320604_0004/job_1455503320604_0004_1.jhist
2016-02-14 23:42:50,712 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved Lnx0.localdomain.com to /default-rack
2016-02-14 23:42:50,957 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved Lnx2.localdomain.com to /default-rack
2016-02-14 23:42:51,179 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved Lnx1.localdomain.com to /default-rack
2016-02-14 23:42:51,184 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1455503320604_0004_m_000000 Task Transitioned from NEW to SCHEDULED
2016-02-14 23:42:51,186 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1455503320604_0004_r_000000 Task Transitioned from NEW to SCHEDULED
2016-02-14 23:42:51,188 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2016-02-14 23:42:51,189 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_r_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2016-02-14 23:42:51,681 INFO [Thread-52] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:<memory:256, vCores:1>
2016-02-14 23:42:51,708 INFO [Thread-52] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: reduceResourceRequest:<memory:512, vCores:1>
2016-02-14 23:42:52,684 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2016-02-14 23:42:52,693 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1455503320604_0004: ask=5 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:768, vCores:1> knownNMs=1
2016-02-14 23:42:52,695 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=<memory:768, vCores:1>
2016-02-14 23:42:52,695 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2016-02-14 23:42:53,765 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2016-02-14 23:42:53,782 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e126_1455503320604_0004_01_000002 to attempt_1455503320604_0004_m_000000_0
2016-02-14 23:42:53,785 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=<memory:512, vCores:1>
2016-02-14 23:42:53,785 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2016-02-14 23:42:53,785 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:42:54,193 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved Lnx1.localdomain.com to /default-rack
2016-02-14 23:42:54,287 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-jar file on the remote FS is hdfs://HDPCA/user/ambari-qa/.staging/job_1455503320604_0004/job.jar
2016-02-14 23:42:54,330 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /user/ambari-qa/.staging/job_1455503320604_0004/job.xml
2016-02-14 23:42:54,361 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #3 tokens and #1 secret keys for NM use for launching container
2016-02-14 23:42:54,361 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 4
2016-02-14 23:42:54,361 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
2016-02-14 23:42:54,628 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2016-02-14 23:42:54,649 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e126_1455503320604_0004_01_000002 taskAttempt attempt_1455503320604_0004_m_000000_0
2016-02-14 23:42:54,653 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1455503320604_0004_m_000000_0
2016-02-14 23:42:54,665 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : Lnx1.localdomain.com:45454
2016-02-14 23:42:54,824 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1455503320604_0004: ask=5 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:512, vCores:1> knownNMs=1
2016-02-14 23:42:54,857 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1455503320604_0004_m_000000_0 : 13562
2016-02-14 23:42:54,859 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1455503320604_0004_m_000000_0] using containerId: [container_e126_1455503320604_0004_01_000002 on NM: [Lnx1.localdomain.com:45454]
2016-02-14 23:42:54,863 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2016-02-14 23:42:54,864 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1455503320604_0004_m_000000 Task Transitioned from SCHEDULED to RUNNING
2016-02-14 23:43:37,602 INFO [Socket Reader #1 for port 41456] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1455503320604_0004 (auth:SIMPLE)
2016-02-14 23:43:37,766 INFO [Socket Reader #1 for port 41456] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1455503320604_0004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2016-02-14 23:43:37,781 INFO [IPC Server handler 14 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1455503320604_0004_m_138538465099778 asked for a task
2016-02-14 23:43:37,790 INFO [IPC Server handler 14 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1455503320604_0004_m_138538465099778 given task: attempt_1455503320604_0004_m_000000_0
2016-02-14 23:43:42,241 INFO [IPC Server handler 14 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1455503320604_0004_m_000000_0 is : 0.0
2016-02-14 23:43:42,678 INFO [IPC Server handler 5 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1455503320604_0004_m_000000_0 is : 1.0
2016-02-14 23:43:42,709 INFO [IPC Server handler 8 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1455503320604_0004_m_000000_0
2016-02-14 23:43:42,715 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_m_000000_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP
2016-02-14 23:43:42,739 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e126_1455503320604_0004_01_000002 taskAttempt attempt_1455503320604_0004_m_000000_0
2016-02-14 23:43:42,739 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1455503320604_0004_m_000000_0
2016-02-14 23:43:42,740 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : Lnx1.localdomain.com:45454
2016-02-14 23:43:42,876 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_m_000000_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED
2016-02-14 23:43:43,020 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1455503320604_0004_m_000000_0
2016-02-14 23:43:43,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1455503320604_0004_m_000000 Task Transitioned from RUNNING to SUCCEEDED
2016-02-14 23:43:43,025 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
2016-02-14 23:43:43,544 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:43:43,550 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=<memory:512, vCores:1>
2016-02-14 23:43:43,550 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold reached. Scheduling reduces.
2016-02-14 23:43:43,550 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: All maps assigned. Ramping up all remaining reduces:1
2016-02-14 23:43:43,550 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:1 AssignedMaps:1 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:43:44,562 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1455503320604_0004: ask=1 release= 0 newContainers=0 finishedContainers=1 resourcelimit=<memory:768, vCores:1> knownNMs=1
2016-02-14 23:43:44,562 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e126_1455503320604_0004_01_000002
2016-02-14 23:43:44,563 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:1 AssignedMaps:0 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:43:44,564 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1455503320604_0004_m_000000_0: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

2016-02-14 23:43:45,574 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2016-02-14 23:43:45,575 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned to reduce
2016-02-14 23:43:45,575 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e126_1455503320604_0004_01_000003 to attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:45,575 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:1 CompletedMaps:1 CompletedReds:0 ContAlloc:2 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:43:45,588 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved Lnx1.localdomain.com to /default-rack
2016-02-14 23:43:45,589 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_r_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2016-02-14 23:43:45,609 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e126_1455503320604_0004_01_000003 taskAttempt attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:45,610 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:45,610 INFO [ContainerLauncher #2] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : Lnx1.localdomain.com:45454
2016-02-14 23:43:45,740 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1455503320604_0004_r_000000_0 : 13562
2016-02-14 23:43:45,741 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1455503320604_0004_r_000000_0] using containerId: [container_e126_1455503320604_0004_01_000003 on NM: [Lnx1.localdomain.com:45454]
2016-02-14 23:43:45,741 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_r_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2016-02-14 23:43:45,742 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1455503320604_0004_r_000000 Task Transitioned from SCHEDULED to RUNNING
2016-02-14 23:43:46,587 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1455503320604_0004: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:256, vCores:1> knownNMs=1
2016-02-14 23:43:52,144 INFO [Socket Reader #1 for port 41456] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1455503320604_0004 (auth:SIMPLE)
2016-02-14 23:43:52,179 INFO [Socket Reader #1 for port 41456] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1455503320604_0004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2016-02-14 23:43:52,181 INFO [IPC Server handler 14 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1455503320604_0004_r_138538465099779 asked for a task
2016-02-14 23:43:52,181 INFO [IPC Server handler 14 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1455503320604_0004_r_138538465099779 given task: attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:55,506 INFO [IPC Server handler 5 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: MapCompletionEvents request from attempt_1455503320604_0004_r_000000_0. startIndex 0 maxEvents 10000
2016-02-14 23:43:56,373 INFO [IPC Server handler 5 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1455503320604_0004_r_000000_0 is : 0.0
2016-02-14 23:43:56,467 INFO [IPC Server handler 8 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1455503320604_0004_r_000000_0 is : 0.0
2016-02-14 23:43:58,565 INFO [IPC Server handler 13 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:58,573 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_r_000000_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2016-02-14 23:43:58,574 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1455503320604_0004_r_000000_0 given a go for committing the task output.
2016-02-14 23:43:58,588 INFO [IPC Server handler 6 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:58,589 INFO [IPC Server handler 6 on 41456] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1455503320604_0004_r_000000_0:true
2016-02-14 23:43:58,836 INFO [IPC Server handler 12 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1455503320604_0004_r_000000_0 is : 1.0
2016-02-14 23:43:58,853 INFO [IPC Server handler 9 on 41456] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:58,854 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_r_000000_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_CONTAINER_CLEANUP
2016-02-14 23:43:58,874 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e126_1455503320604_0004_01_000003 taskAttempt attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:58,875 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:58,875 INFO [ContainerLauncher #3] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : Lnx1.localdomain.com:45454
2016-02-14 23:43:58,952 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1455503320604_0004_r_000000_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED
2016-02-14 23:43:58,953 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1455503320604_0004_r_000000_0
2016-02-14 23:43:58,953 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1455503320604_0004_r_000000 Task Transitioned from RUNNING to SUCCEEDED
2016-02-14 23:43:58,953 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 2
2016-02-14 23:43:58,953 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1455503320604_0004Job Transitioned from RUNNING to COMMITTING
2016-02-14 23:43:59,040 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT
2016-02-14 23:43:59,736 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:1 CompletedMaps:1 CompletedReds:1 ContAlloc:2 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:43:59,788 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent 
2016-02-14 23:43:59,789 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1455503320604_0004Job Transitioned from COMMITTING to SUCCEEDED
2016-02-14 23:43:59,825 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry
2016-02-14 23:43:59,825 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true
2016-02-14 23:43:59,825 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified that shouldUnregistered is: true
2016-02-14 23:43:59,827 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2016-02-14 23:43:59,827 INFO [Thread-74] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true
2016-02-14 23:43:59,827 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services
2016-02-14 23:43:59,829 INFO [Thread-74] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
2016-02-14 23:44:00,258 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://HDPCA:8020/user/ambari-qa/.staging/job_1455503320604_0004/job_1455503320604_0004_1.jhist to hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004-1455507795584-ambari%2Dqa-word+count-1455511439776-1-1-SUCCEEDED-default-1455511369353.jhist_tmp
2016-02-14 23:44:00,764 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e126_1455503320604_0004_01_000003
2016-02-14 23:44:00,764 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:1 CompletedReds:1 ContAlloc:2 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:44:00,764 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1455503320604_0004_r_000000_0: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

2016-02-14 23:44:01,395 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004-1455507795584-ambari%2Dqa-word+count-1455511439776-1-1-SUCCEEDED-default-1455511369353.jhist_tmp
2016-02-14 23:44:01,465 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://HDPCA:8020/user/ambari-qa/.staging/job_1455503320604_0004/job_1455503320604_0004_1_conf.xml to hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004_conf.xml_tmp
2016-02-14 23:44:03,117 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004_conf.xml_tmp
2016-02-14 23:44:03,175 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004.summary_tmp to hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004.summary
2016-02-14 23:44:03,210 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004_conf.xml_tmp to hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004_conf.xml
2016-02-14 23:44:03,243 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004-1455507795584-ambari%2Dqa-word+count-1455511439776-1-1-SUCCEEDED-default-1455511369353.jhist_tmp to hdfs://HDPCA:8020/mr-history/tmp/ambari-qa/job_1455503320604_0004-1455507795584-ambari%2Dqa-word+count-1455511439776-1-1-SUCCEEDED-default-1455511369353.jhist
2016-02-14 23:44:03,244 INFO [Thread-74] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2016-02-14 23:44:03,248 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to 
2016-02-14 23:44:03,249 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://Lnx1.localdomain.com:19888/jobhistory/job/job_1455503320604_0004
2016-02-14 23:44:03,459 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.
2016-02-14 23:44:04,463 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:1 CompletedReds:1 ContAlloc:2 ContRel:0 HostLocal:1 RackLocal:0
2016-02-14 23:44:04,465 INFO [Thread-74] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://HDPCA /user/ambari-qa/.staging/job_1455503320604_0004
2016-02-14 23:44:04,500 INFO [Thread-74] org.apache.hadoop.ipc.Server: Stopping server on 41456
2016-02-14 23:44:04,968 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2016-02-14 23:44:04,968 INFO [IPC Server listener on 41456] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 41456
2016-02-14 23:44:04,969 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted