Log Type: syslog

Log Upload Time: Wed Jul 31 17:36:26 +0200 2019

Log Length: 26170

2019-07-31 17:36:12,907 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1547225966242_3376_000001
2019-07-31 17:36:13,010 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-07-31 17:36:13,027 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration
2019-07-31 17:36:13,033 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2019-07-31 17:36:13,033 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 3376 cluster_timestamp: 1547225966242 } attemptId: 1 } keyId: 250917431)
2019-07-31 17:36:13,170 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: RM_DELEGATION_TOKEN, Service: 10.65.12.141:8032,10.65.12.142:8032, Ident: (owner=whorchani, renewer=oozie mr token, realUser=oozie, issueDate=1564587371165, maxDate=1565192171165, sequenceNumber=92189, masterKeyId=997)
2019-07-31 17:36:13,181 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2019-07-31 17:36:13,181 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2019-07-31 17:36:13,569 WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2019-07-31 17:36:13,670 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2019-07-31 17:36:13,671 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2019-07-31 17:36:13,671 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2019-07-31 17:36:13,671 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2019-07-31 17:36:13,671 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2019-07-31 17:36:13,672 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2019-07-31 17:36:13,672 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2019-07-31 17:36:13,672 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2019-07-31 17:36:13,691 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore -  ignoring
2019-07-31 17:36:13,704 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore -  ignoring
2019-07-31 17:36:13,715 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system is set solely by core-default.xml therefore -  ignoring
2019-07-31 17:36:13,745 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2019-07-31 17:36:13,763 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2019-07-31 17:36:13,795 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-mrappmaster.properties,hadoop-metrics2.properties
2019-07-31 17:36:13,828 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2019-07-31 17:36:13,828 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2019-07-31 17:36:13,833 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1547225966242_3376 to jobTokenSecretManager
2019-07-31 17:36:13,917 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1547225966242_3376 because: not enabled;
2019-07-31 17:36:13,926 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1547225966242_3376 = 0. Number of splits = 1
2019-07-31 17:36:13,926 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1547225966242_3376 = 0
2019-07-31 17:36:13,926 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1547225966242_3376Job Transitioned from NEW to INITED
2019-07-31 17:36:13,927 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1547225966242_3376.
2019-07-31 17:36:13,939 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2019-07-31 17:36:13,945 INFO [Socket Reader #1 for port 38519] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 38519
2019-07-31 17:36:13,972 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2019-07-31 17:36:13,972 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2019-07-31 17:36:13,973 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at worker6.hdp.bdt.cf-infra.net/10.65.12.115:38519
2019-07-31 17:36:13,973 INFO [IPC Server listener on 38519] org.apache.hadoop.ipc.Server: IPC Server listener on 38519: starting
2019-07-31 17:36:14,019 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019-07-31 17:36:14,023 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2019-07-31 17:36:14,026 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2019-07-31 17:36:14,029 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2019-07-31 17:36:14,050 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2019-07-31 17:36:14,050 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2019-07-31 17:36:14,051 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2019-07-31 17:36:14,052 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2019-07-31 17:36:14,187 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2019-07-31 17:36:14,188 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 55420
2019-07-31 17:36:14,188 INFO [main] org.mortbay.log: jetty-6.1.26.hwx
2019-07-31 17:36:14,205 INFO [main] org.mortbay.log: Extract jar:file:/hadoop/yarn/local/filecache/1382/mapreduce.tar.gz/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar!/webapps/mapreduce to ./tmp/Jetty_0_0_0_0_55420_mapreduce____gwez3b/webapp
2019-07-31 17:36:14,808 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:55420
2019-07-31 17:36:14,808 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 55420
2019-07-31 17:36:14,810 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2019-07-31 17:36:14,811 INFO [Socket Reader #1 for port 41991] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 41991
2019-07-31 17:36:14,815 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2019-07-31 17:36:14,815 INFO [IPC Server listener on 41991] org.apache.hadoop.ipc.Server: IPC Server listener on 41991: starting
2019-07-31 17:36:14,839 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2019-07-31 17:36:14,839 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2019-07-31 17:36:14,839 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2019-07-31 17:36:14,881 INFO [main] org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider: Looking for the active RM in [rm1, rm2]...
2019-07-31 17:36:14,927 INFO [main] org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider: Found active RM [rm2]
2019-07-31 17:36:14,929 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: <memory:77824, vCores:3>
2019-07-31 17:36:14,929 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: default
2019-07-31 17:36:14,931 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2019-07-31 17:36:14,931 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10
2019-07-31 17:36:14,932 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
2019-07-31 17:36:14,934 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1547225966242_3376Job Transitioned from INITED to SETUP
2019-07-31 17:36:14,935 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2019-07-31 17:36:14,937 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2019-07-31 17:36:14,937 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2019-07-31 17:36:14,948 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1547225966242_3376Job Transitioned from SETUP to RUNNING
2019-07-31 17:36:14,959 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1547225966242_3376_m_000000 Task Transitioned from NEW to SCHEDULED
2019-07-31 17:36:14,959 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1547225966242_3376_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2019-07-31 17:36:14,960 INFO [Thread-52] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:<memory:19456, vCores:1>
2019-07-31 17:36:14,977 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1547225966242_3376, File: hdfs://hdfsclusterprod:8020/user/whorchani/.staging/job_1547225966242_3376/job_1547225966242_3376_1.jhist
2019-07-31 17:36:15,930 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2019-07-31 17:36:15,955 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1547225966242_3376: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:561152, vCores:1> knownNMs=7
2019-07-31 17:36:16,961 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2019-07-31 17:36:16,973 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved worker2.hdp.bdt.cf-infra.net to /default-rack
2019-07-31 17:36:16,974 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e111_1547225966242_3376_01_000002 to attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:16,974 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
2019-07-31 17:36:16,998 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved worker2.hdp.bdt.cf-infra.net to /default-rack
2019-07-31 17:36:16,998 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Job jar is not present. Not adding any jar to the list of resources.
2019-07-31 17:36:17,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /user/whorchani/.staging/job_1547225966242_3376/job.xml
2019-07-31 17:36:17,161 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #1 tokens and #1 secret keys for NM use for launching container
2019-07-31 17:36:17,161 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 2
2019-07-31 17:36:17,161 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
2019-07-31 17:36:17,297 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1547225966242_3376_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2019-07-31 17:36:17,300 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e111_1547225966242_3376_01_000002 taskAttempt attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:17,301 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:17,301 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : worker2.hdp.bdt.cf-infra.net:45454
2019-07-31 17:36:17,338 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1547225966242_3376_m_000000_0 : 13562
2019-07-31 17:36:17,339 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1547225966242_3376_m_000000_0] using containerId: [container_e111_1547225966242_3376_01_000002 on NM: [worker2.hdp.bdt.cf-infra.net:45454]
2019-07-31 17:36:17,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1547225966242_3376_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2019-07-31 17:36:17,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1547225966242_3376_m_000000 Task Transitioned from SCHEDULED to RUNNING
2019-07-31 17:36:17,976 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1547225966242_3376: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:541696, vCores:1> knownNMs=7
2019-07-31 17:36:18,241 INFO [Socket Reader #1 for port 41991] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1547225966242_3376 (auth:SIMPLE)
2019-07-31 17:36:18,252 INFO [IPC Server handler 0 on 41991] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1547225966242_3376_m_122045790683138 asked for a task
2019-07-31 17:36:18,252 INFO [IPC Server handler 0 on 41991] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1547225966242_3376_m_122045790683138 given task: attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:19,129 INFO [IPC Server handler 0 on 41991] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1547225966242_3376_m_000000_0 is : 0.0
2019-07-31 17:36:19,150 INFO [IPC Server handler 1 on 41991] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:19,150 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1547225966242_3376_m_000000_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2019-07-31 17:36:19,151 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1547225966242_3376_m_000000_0 given a go for committing the task output.
2019-07-31 17:36:19,151 INFO [IPC Server handler 3 on 41991] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:19,152 INFO [IPC Server handler 3 on 41991] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1547225966242_3376_m_000000_0:true
2019-07-31 17:36:19,171 INFO [IPC Server handler 2 on 41991] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1547225966242_3376_m_000000_0 is : 1.0
2019-07-31 17:36:19,176 INFO [IPC Server handler 4 on 41991] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:19,176 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1547225966242_3376_m_000000_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_CONTAINER_CLEANUP
2019-07-31 17:36:19,177 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e111_1547225966242_3376_01_000002 taskAttempt attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:19,177 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:19,177 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : worker2.hdp.bdt.cf-infra.net:45454
2019-07-31 17:36:19,216 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1547225966242_3376_m_000000_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED
2019-07-31 17:36:19,220 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1547225966242_3376_m_000000_0
2019-07-31 17:36:19,220 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1547225966242_3376_m_000000 Task Transitioned from RUNNING to SUCCEEDED
2019-07-31 17:36:19,221 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
2019-07-31 17:36:19,222 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1547225966242_3376Job Transitioned from RUNNING to COMMITTING
2019-07-31 17:36:19,222 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT
2019-07-31 17:36:19,283 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent 
2019-07-31 17:36:19,283 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1547225966242_3376Job Transitioned from COMMITTING to SUCCEEDED
2019-07-31 17:36:19,284 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry
2019-07-31 17:36:19,284 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true
2019-07-31 17:36:19,284 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified that shouldUnregistered is: true
2019-07-31 17:36:19,284 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2019-07-31 17:36:19,284 INFO [Thread-70] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true
2019-07-31 17:36:19,284 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services
2019-07-31 17:36:19,284 INFO [Thread-70] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
2019-07-31 17:36:19,326 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://hdfsclusterprod:8020/user/whorchani/.staging/job_1547225966242_3376/job_1547225966242_3376_1.jhist to hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376-1564587371972-whorchani-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell_3%3AA%3Ds-1564587379282-1-0-SUCCEEDED-default-1564587374933.jhist_tmp
2019-07-31 17:36:19,347 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376-1564587371972-whorchani-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell_3%3AA%3Ds-1564587379282-1-0-SUCCEEDED-default-1564587374933.jhist_tmp
2019-07-31 17:36:19,353 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://hdfsclusterprod:8020/user/whorchani/.staging/job_1547225966242_3376/job_1547225966242_3376_1_conf.xml to hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376_conf.xml_tmp
2019-07-31 17:36:19,385 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376_conf.xml_tmp
2019-07-31 17:36:19,411 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376.summary_tmp to hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376.summary
2019-07-31 17:36:19,418 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376_conf.xml_tmp to hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376_conf.xml
2019-07-31 17:36:19,421 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376-1564587371972-whorchani-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell_3%3AA%3Ds-1564587379282-1-0-SUCCEEDED-default-1564587374933.jhist_tmp to hdfs://hdfsclusterprod:8020/mr-history/tmp/whorchani/job_1547225966242_3376-1564587371972-whorchani-oozie%3Alauncher%3AT%3Dshell%3AW%3Dshell_3%3AA%3Ds-1564587379282-1-0-SUCCEEDED-default-1564587374933.jhist
2019-07-31 17:36:19,421 INFO [Thread-70] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2019-07-31 17:36:19,422 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to 
2019-07-31 17:36:19,422 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://utility1.hdp.bdt.cf-infra.net:19888/jobhistory/job/job_1547225966242_3376
2019-07-31 17:36:19,427 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.
2019-07-31 17:36:20,428 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
2019-07-31 17:36:20,429 INFO [Thread-70] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://hdfsclusterprod /user/whorchani/.staging/job_1547225966242_3376
2019-07-31 17:36:20,431 INFO [Thread-70] org.apache.hadoop.ipc.Server: Stopping server on 41991
2019-07-31 17:36:20,432 INFO [IPC Server listener on 41991] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 41991
2019-07-31 17:36:20,432 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
2019-07-31 17:36:20,432 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder