Hi @Pierre Villard , I restarted App TimeLine Server and ResourceManager and that solved the problem. however, here is log of yarn node manager. 2016-05-26 05:47:55,939 INFO yarn.YarnShuffleService (YarnShuffleService.java:initializeContainer(183)) - Initializing container container_1464255636652_0012_01_000001 2016-05-26 05:47:55,943 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hadoop-mapreduce-client-core-2.7.1.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,943 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-server-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,943 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/netty-all-4.0.23.Final.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/protobuf-java-2.5.0.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hadoop-common-2.7.1.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/htrace-core-3.1.0-incubating.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-client-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/metrics-core-2.2.0.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/zookeeper-3.4.6.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-protocol-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-common-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/guava-12.0.1.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.split transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.splitmetainfo transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.xml transitioned from INIT to DOWNLOADING 2016-05-26 05:47:55,944 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(711)) - Created localizer for container_1464255636652_0012_01_000001 2016-05-26 05:47:55,945 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1191)) - Writing credentials to the nmPrivate file /hadoop/hdfs1/hadoop/yarn/local/nmPrivate/container_1464255636652_0012_01_000001.tokens. Credentials list: 2016-05-26 05:47:55,967 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(610)) - Initializing user hdfs 2016-05-26 05:47:55,968 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(117)) - Copying from /hadoop/hdfs1/hadoop/yarn/local/nmPrivate/container_1464255636652_0012_01_000001.tokens to /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000001.tokens 2016-05-26 05:47:55,968 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(124)) - Localizer CWD set to /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012 = file:/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012 2016-05-26 05:47:56,008 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hadoop-mapreduce-client-core-2.7.1.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4052/hadoop-mapreduce-client-core-2.7.1.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,033 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-server-1.1.2.2.4.0.0-169.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4053/hbase-server-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,052 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/netty-all-4.0.23.Final.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4054/netty-all-4.0.23.Final.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,068 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/protobuf-java-2.5.0.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4055/protobuf-java-2.5.0.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,093 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hadoop-common-2.7.1.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4056/hadoop-common-2.7.1.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,111 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/htrace-core-3.1.0-incubating.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4057/htrace-core-3.1.0-incubating.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,135 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-client-1.1.2.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4058/hbase-client-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,150 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/metrics-core-2.2.0.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4059/metrics-core-2.2.0.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,167 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/zookeeper-3.4.6.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4060/zookeeper-3.4.6.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,192 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-protocol-1.1.2.2.4.0.0-169.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4061/hbase-protocol-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,208 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-common-1.1.2.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4062/hbase-common-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,223 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4063/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,242 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/libjars/guava-12.0.1.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4064/guava-12.0.1.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,260 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.split(->/var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/filecache/10/job.split) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,276 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.splitmetainfo(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/filecache/11/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,302 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/filecache/12/job.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,318 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0012/job.xml(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/filecache/13/job.xml) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:47:56,318 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000001 transitioned from LOCALIZING to LOCALIZED 2016-05-26 05:47:56,336 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000001 transitioned from LOCALIZED to RUNNING 2016-05-26 05:47:56,338 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(268)) - launchContainer: [bash, /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000001/default_container_executor.sh] 2016-05-26 05:47:57,647 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(375)) - Starting resource-monitoring for container_1464255636652_0012_01_000001 2016-05-26 05:47:57,657 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 25938 for container-id container_1464255636652_0012_01_000001: 142.0 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used 2016-05-26 05:48:00,686 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 25938 for container-id container_1464255636652_0012_01_000001: 341.4 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used 2016-05-26 05:48:03,324 INFO ipc.Server (Server.java:saslProcess(1386)) - Auth successful for appattempt_1464255636652_0012_000001 (auth:SIMPLE) 2016-05-26 05:48:03,330 INFO containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(816)) - Start request for container_1464255636652_0012_01_000002 by user hdfs 2016-05-26 05:48:03,331 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs IP=10.1.10.204 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1464255636652_0012 CONTAINERID=container_1464255636652_0012_01_000002 2016-05-26 05:48:03,331 INFO application.ApplicationImpl (ApplicationImpl.java:transition(304)) - Adding container_1464255636652_0012_01_000002 to application application_1464255636652_0012 2016-05-26 05:48:03,331 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000002 transitioned from NEW to LOCALIZING 2016-05-26 05:48:03,332 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1464255636652_0012 2016-05-26 05:48:03,332 INFO yarn.YarnShuffleService (YarnShuffleService.java:initializeContainer(183)) - Initializing container container_1464255636652_0012_01_000002 2016-05-26 05:48:03,332 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_INIT for appId application_1464255636652_0012 2016-05-26 05:48:03,332 INFO containermanager.AuxServices (AuxServices.java:handle(200)) - Got APPLICATION_INIT for service mapreduce_shuffle 2016-05-26 05:48:03,332 INFO mapred.ShuffleHandler (ShuffleHandler.java:addJobToken(671)) - Added token for job_1464255636652_0012 2016-05-26 05:48:03,332 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/job.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:03,332 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/job.xml transitioned from INIT to DOWNLOADING 2016-05-26 05:48:03,333 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(711)) - Created localizer for container_1464255636652_0012_01_000002 2016-05-26 05:48:03,334 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1191)) - Writing credentials to the nmPrivate file /hadoop/hdfs1/hadoop/yarn/local/nmPrivate/container_1464255636652_0012_01_000002.tokens. Credentials list: 2016-05-26 05:48:03,380 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(610)) - Initializing user hdfs 2016-05-26 05:48:03,381 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(117)) - Copying from /hadoop/hdfs1/hadoop/yarn/local/nmPrivate/container_1464255636652_0012_01_000002.tokens to /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000002.tokens 2016-05-26 05:48:03,381 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(124)) - Localizer CWD set to /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012 = file:/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012 2016-05-26 05:48:03,435 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/job.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/filecache/14/job.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:03,456 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0012/job.xml(->/var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/filecache/15/job.xml) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:03,456 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000002 transitioned from LOCALIZING to LOCALIZED 2016-05-26 05:48:03,478 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000002 transitioned from LOCALIZED to RUNNING 2016-05-26 05:48:03,481 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(268)) - launchContainer: [bash, /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000002/default_container_executor.sh] 2016-05-26 05:48:03,686 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(375)) - Starting resource-monitoring for container_1464255636652_0012_01_000002 2016-05-26 05:48:03,701 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 25938 for container-id container_1464255636652_0012_01_000001: 395.1 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:03,723 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 26129 for container-id container_1464255636652_0012_01_000002: 43.3 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used 2016-05-26 05:48:06,474 INFO ipc.Server (Server.java:saslProcess(1386)) - Auth successful for appattempt_1464255636652_0012_000001 (auth:SIMPLE) 2016-05-26 05:48:06,478 INFO containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(966)) - Stopping container with container Id: container_1464255636652_0012_01_000002 2016-05-26 05:48:06,478 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs IP=10.1.10.204 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1464255636652_0012 CONTAINERID=container_1464255636652_0012_01_000002 2016-05-26 05:48:06,478 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000002 transitioned from RUNNING to KILLING 2016-05-26 05:48:06,478 INFO launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(371)) - Cleaning up container container_1464255636652_0012_01_000002 2016-05-26 05:48:06,502 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(224)) - Exit code from container container_1464255636652_0012_01_000002 is : 143 2016-05-26 05:48:06,509 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 2016-05-26 05:48:06,509 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1464255636652_0012 CONTAINERID=container_1464255636652_0012_01_000002 2016-05-26 05:48:06,510 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000002 2016-05-26 05:48:06,510 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000002 2016-05-26 05:48:06,511 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 2016-05-26 05:48:06,511 INFO application.ApplicationImpl (ApplicationImpl.java:transition(347)) - Removing container_1464255636652_0012_01_000002 from application application_1464255636652_0012 2016-05-26 05:48:06,511 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:startContainerLogAggregation(547)) - Considering container container_1464255636652_0012_01_000002 for log-aggregation 2016-05-26 05:48:06,511 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1464255636652_0012 2016-05-26 05:48:06,511 INFO yarn.YarnShuffleService (YarnShuffleService.java:stopContainer(189)) - Stopping container container_1464255636652_0012_01_000002 2016-05-26 05:48:06,724 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(390)) - Stopping resource-monitoring for container_1464255636652_0012_01_000002 2016-05-26 05:48:06,733 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 25938 for container-id container_1464255636652_0012_01_000001: 411.9 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:09,746 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 25938 for container-id container_1464255636652_0012_01_000001: 412.8 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:12,755 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 25938 for container-id container_1464255636652_0012_01_000001: 412.8 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:12,912 INFO launcher.ContainerLaunch (ContainerLaunch.java:call(347)) - Container container_1464255636652_0012_01_000001 succeeded 2016-05-26 05:48:12,913 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2016-05-26 05:48:12,913 INFO launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(371)) - Cleaning up container container_1464255636652_0012_01_000001 2016-05-26 05:48:12,935 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000001 2016-05-26 05:48:12,935 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012/container_1464255636652_0012_01_000001 2016-05-26 05:48:12,935 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs OPERATION=Container Finished - SucceededTARGET=ContainerImpl RESULT=SUCCESS APPID=application_1464255636652_0012 CONTAINERID=container_1464255636652_0012_01_000001 2016-05-26 05:48:12,936 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0012_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2016-05-26 05:48:12,936 INFO application.ApplicationImpl (ApplicationImpl.java:transition(347)) - Removing container_1464255636652_0012_01_000001 from application application_1464255636652_0012 2016-05-26 05:48:12,936 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:startContainerLogAggregation(547)) - Considering container container_1464255636652_0012_01_000001 for log-aggregation 2016-05-26 05:48:12,936 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1464255636652_0012 2016-05-26 05:48:12,936 INFO yarn.YarnShuffleService (YarnShuffleService.java:stopContainer(189)) - Stopping container container_1464255636652_0012_01_000001 2016-05-26 05:48:13,495 INFO ipc.Server (Server.java:saslProcess(1386)) - Auth successful for appattempt_1464255636652_0012_000001 (auth:SIMPLE) 2016-05-26 05:48:13,499 INFO containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(966)) - Stopping container with container Id: container_1464255636652_0012_01_000001 2016-05-26 05:48:13,500 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs IP=10.1.10.20 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1464255636652_0012 CONTAINERID=container_1464255636652_0012_01_000001 2016-05-26 05:48:13,500 INFO nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(529)) - Removed completed containers from NM context: [container_1464255636652_0012_01_000001] 2016-05-26 05:48:13,501 INFO application.ApplicationImpl (ApplicationImpl.java:handle(464)) - Application application_1464255636652_0012 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2016-05-26 05:48:13,501 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1464255636652_0012 2016-05-26 05:48:13,501 INFO yarn.YarnShuffleService (YarnShuffleService.java:stopApplication(170)) - Stopping application application_1464255636652_0012 2016-05-26 05:48:13,501 INFO shuffle.ExternalShuffleBlockResolver (ExternalShuffleBlockResolver.java:applicationRemoved(206)) - Application application_1464255636652_0012 removed, cleanupLocalDirs = false 2016-05-26 05:48:13,501 INFO application.ApplicationImpl (ApplicationImpl.java:handle(464)) - Application application_1464255636652_0012 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2016-05-26 05:48:13,501 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:finishLogAggregation(555)) - Application just finished : application_1464255636652_0012 2016-05-26 05:48:13,502 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012 2016-05-26 05:48:13,503 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0012 2016-05-26 05:48:13,513 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:doContainerLogAggregation(602)) - Uploading logs for container container_1464255636652_0012_01_000001. Current good log dirs are /var/log/hadoop/yarn/log,/hadoop/hdfs1/hadoop/yarn/log 2016-05-26 05:48:13,514 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:doContainerLogAggregation(602)) - Uploading logs for container container_1464255636652_0012_01_000002. Current good log dirs are /var/log/hadoop/yarn/log,/hadoop/hdfs1/hadoop/yarn/log 2016-05-26 05:48:13,514 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000001/directory.info 2016-05-26 05:48:13,514 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000001/launch_container.sh 2016-05-26 05:48:13,514 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000001/stderr 2016-05-26 05:48:13,515 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000001/stdout 2016-05-26 05:48:13,515 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000001/syslog 2016-05-26 05:48:13,516 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000002/stderr 2016-05-26 05:48:13,516 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000002/directory.info 2016-05-26 05:48:13,516 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000002/syslog 2016-05-26 05:48:13,516 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000002/launch_container.sh 2016-05-26 05:48:13,516 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0012/container_1464255636652_0012_01_000002/stdout 2016-05-26 05:48:13,550 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0012 2016-05-26 05:48:13,551 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0012 2016-05-26 05:48:14,520 INFO ipc.Server (Server.java:saslProcess(1386)) - Auth successful for appattempt_1464255636652_0013_000001 (auth:SIMPLE) 2016-05-26 05:48:14,522 INFO containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(816)) - Start request for container_1464255636652_0013_01_000001 by user hdfs 2016-05-26 05:48:14,523 INFO containermanager.ContainerManagerImpl (ContainerManagerImpl.java:startContainerInternal(856)) - Creating a new application reference for app application_1464255636652_0013 2016-05-26 05:48:14,523 INFO application.ApplicationImpl (ApplicationImpl.java:handle(464)) - Application application_1464255636652_0013 transitioned from NEW to INITING 2016-05-26 05:48:14,523 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs IP=10.1.10.20 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1464255636652_0013 CONTAINERID=container_1464255636652_0013_01_000001 2016-05-26 05:48:14,524 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2016-05-26 05:48:14,525 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:<init>(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2016-05-26 05:48:14,543 INFO application.ApplicationImpl (ApplicationImpl.java:transition(304)) - Adding container_1464255636652_0013_01_000001 to application application_1464255636652_0013 2016-05-26 05:48:14,543 INFO application.ApplicationImpl (ApplicationImpl.java:handle(464)) - Application application_1464255636652_0013 transitioned from INITING to RUNNING 2016-05-26 05:48:14,543 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0013_01_000001 transitioned from NEW to LOCALIZING 2016-05-26 05:48:14,544 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_INIT for appId application_1464255636652_0013 2016-05-26 05:48:14,544 INFO yarn.YarnShuffleService (YarnShuffleService.java:initializeContainer(183)) - Initializing container container_1464255636652_0013_01_000001 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hadoop-mapreduce-client-core-2.7.1.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-server-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/netty-all-4.0.23.Final.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/protobuf-java-2.5.0.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hadoop-common-2.7.1.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/htrace-core-3.1.0-incubating.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-client-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/metrics-core-2.2.0.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/zookeeper-3.4.6.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-protocol-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-common-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/guava-12.0.1.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.split transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.splitmetainfo transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.jar transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.xml transitioned from INIT to DOWNLOADING 2016-05-26 05:48:14,544 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:handle(711)) - Created localizer for container_1464255636652_0013_01_000001 2016-05-26 05:48:14,545 INFO localizer.ResourceLocalizationService (ResourceLocalizationService.java:writeCredentials(1191)) - Writing credentials to the nmPrivate file /hadoop/hdfs1/hadoop/yarn/local/nmPrivate/container_1464255636652_0013_01_000001.tokens. Credentials list: 2016-05-26 05:48:14,568 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:createUserCacheDirs(610)) - Initializing user hdfs 2016-05-26 05:48:14,569 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(117)) - Copying from /hadoop/hdfs1/hadoop/yarn/local/nmPrivate/container_1464255636652_0013_01_000001.tokens to /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/container_1464255636652_0013_01_000001.tokens 2016-05-26 05:48:14,569 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:startLocalizer(124)) - Localizer CWD set to /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013 = file:/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013 2016-05-26 05:48:14,613 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hadoop-mapreduce-client-core-2.7.1.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4065/hadoop-mapreduce-client-core-2.7.1.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,638 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-server-1.1.2.2.4.0.0-169.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4066/hbase-server-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,657 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/netty-all-4.0.23.Final.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4067/netty-all-4.0.23.Final.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,675 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/protobuf-java-2.5.0.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4068/protobuf-java-2.5.0.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,698 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hadoop-common-2.7.1.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4069/hadoop-common-2.7.1.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,716 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/htrace-core-3.1.0-incubating.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4070/htrace-core-3.1.0-incubating.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,734 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-client-1.1.2.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4071/hbase-client-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,748 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/metrics-core-2.2.0.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4072/metrics-core-2.2.0.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,765 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/zookeeper-3.4.6.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4073/zookeeper-3.4.6.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,790 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-protocol-1.1.2.2.4.0.0-169.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4074/hbase-protocol-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,810 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-common-1.1.2.2.4.0.0-169.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4075/hbase-common-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,828 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/filecache/4076/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,847 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster/user/hdfs/.staging/job_1464255636652_0013/libjars/guava-12.0.1.jar(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/filecache/4077/guava-12.0.1.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,864 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.split(->/var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/filecache/10/job.split) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,880 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.splitmetainfo(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/filecache/11/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,912 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.jar(->/var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/filecache/12/job.jar) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,930 INFO localizer.LocalizedResource (LocalizedResource.java:handle(203)) - Resource hdfs://aimprodcluster:8020/user/hdfs/.staging/job_1464255636652_0013/job.xml(->/hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/filecache/13/job.xml) transitioned from DOWNLOADING to LOCALIZED 2016-05-26 05:48:14,930 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0013_01_000001 transitioned from LOCALIZING to LOCALIZED 2016-05-26 05:48:14,955 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0013_01_000001 transitioned from LOCALIZED to RUNNING 2016-05-26 05:48:14,959 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:buildCommandExecutor(268)) - launchContainer: [bash, /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/container_1464255636652_0013_01_000001/default_container_executor.sh] 2016-05-26 05:48:15,757 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(375)) - Starting resource-monitoring for container_1464255636652_0013_01_000001 2016-05-26 05:48:15,757 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(390)) - Stopping resource-monitoring for container_1464255636652_0012_01_000001 2016-05-26 05:48:15,787 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 26444 for container-id container_1464255636652_0013_01_000001: 108.4 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used 2016-05-26 05:48:18,801 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 26444 for container-id container_1464255636652_0013_01_000001: 316.7 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used 2016-05-26 05:48:21,809 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 26444 for container-id container_1464255636652_0013_01_000001: 349.4 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:24,823 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 26444 for container-id container_1464255636652_0013_01_000001: 384.5 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:27,832 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 26444 for container-id container_1464255636652_0013_01_000001: 410.3 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:30,841 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(464)) - Memory usage of ProcessTree 26444 for container-id container_1464255636652_0013_01_000001: 410.3 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used 2016-05-26 05:48:32,057 INFO launcher.ContainerLaunch (ContainerLaunch.java:call(347)) - Container container_1464255636652_0013_01_000001 succeeded 2016-05-26 05:48:32,057 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0013_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 2016-05-26 05:48:32,057 INFO launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(371)) - Cleaning up container container_1464255636652_0013_01_000001 2016-05-26 05:48:32,079 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/container_1464255636652_0013_01_000001 2016-05-26 05:48:32,079 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013/container_1464255636652_0013_01_000001 2016-05-26 05:48:32,079 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs OPERATION=Container Finished - SucceededTARGET=ContainerImpl RESULT=SUCCESS APPID=application_1464255636652_0013 CONTAINERID=container_1464255636652_0013_01_000001 2016-05-26 05:48:32,080 INFO container.ContainerImpl (ContainerImpl.java:handle(1131)) - Container container_1464255636652_0013_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 2016-05-26 05:48:32,080 INFO application.ApplicationImpl (ApplicationImpl.java:transition(347)) - Removing container_1464255636652_0013_01_000001 from application application_1464255636652_0013 2016-05-26 05:48:32,080 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:startContainerLogAggregation(547)) - Considering container container_1464255636652_0013_01_000001 for log-aggregation 2016-05-26 05:48:32,080 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event CONTAINER_STOP for appId application_1464255636652_0013 2016-05-26 05:48:32,080 INFO yarn.YarnShuffleService (YarnShuffleService.java:stopContainer(189)) - Stopping container container_1464255636652_0013_01_000001 2016-05-26 05:48:32,550 INFO ipc.Server (Server.java:saslProcess(1386)) - Auth successful for appattempt_1464255636652_0013_000001 (auth:SIMPLE) 2016-05-26 05:48:32,553 INFO containermanager.ContainerManagerImpl (ContainerManagerImpl.java:stopContainerInternal(966)) - Stopping container with container Id: container_1464255636652_0013_01_000001 2016-05-26 05:48:32,553 INFO nodemanager.NMAuditLogger (NMAuditLogger.java:logSuccess(89)) - USER=hdfs IP=10.1.10.20 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1464255636652_0013 CONTAINERID=container_1464255636652_0013_01_000001 2016-05-26 05:48:32,556 INFO nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:removeOrTrackCompletedContainersFromContext(529)) - Removed completed containers from NM context: [container_1464255636652_0013_01_000001] 2016-05-26 05:48:32,556 INFO application.ApplicationImpl (ApplicationImpl.java:handle(464)) - Application application_1464255636652_0013 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2016-05-26 05:48:32,557 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /var/log/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013 2016-05-26 05:48:32,557 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(458)) - Deleting absolute path : /hadoop/hdfs1/hadoop/yarn/local/usercache/hdfs/appcache/application_1464255636652_0013 2016-05-26 05:48:32,557 INFO containermanager.AuxServices (AuxServices.java:handle(196)) - Got event APPLICATION_STOP for appId application_1464255636652_0013 2016-05-26 05:48:32,557 INFO yarn.YarnShuffleService (YarnShuffleService.java:stopApplication(170)) - Stopping application application_1464255636652_0013 2016-05-26 05:48:32,557 INFO shuffle.ExternalShuffleBlockResolver (ExternalShuffleBlockResolver.java:applicationRemoved(206)) - Application application_1464255636652_0013 removed, cleanupLocalDirs = false 2016-05-26 05:48:32,557 INFO application.ApplicationImpl (ApplicationImpl.java:handle(464)) - Application application_1464255636652_0013 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2016-05-26 05:48:32,557 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:finishLogAggregation(555)) - Application just finished : application_1464255636652_0013 2016-05-26 05:48:32,569 INFO logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:doContainerLogAggregation(602)) - Uploading logs for container container_1464255636652_0013_01_000001. Current good log dirs are /var/log/hadoop/yarn/log,/hadoop/hdfs1/hadoop/yarn/log 2016-05-26 05:48:32,569 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0013/container_1464255636652_0013_01_000001/stdout 2016-05-26 05:48:32,570 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0013/container_1464255636652_0013_01_000001/launch_container.sh 2016-05-26 05:48:32,570 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0013/container_1464255636652_0013_01_000001/directory.info 2016-05-26 05:48:32,570 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0013/container_1464255636652_0013_01_000001/syslog 2016-05-26 05:48:32,570 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0013/container_1464255636652_0013_01_000001/stderr 2016-05-26 05:48:32,606 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /var/log/hadoop/yarn/log/application_1464255636652_0013 2016-05-26 05:48:32,607 INFO nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(467)) - Deleting path : /hadoop/hdfs1/hadoop/yarn/log/application_1464255636652_0013 2016-05-26 05:48:33,841 INFO monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:run(390)) - Stopping resource-monitoring for container_1464255636652_0013_01_000001 Thanks Raja ray ... View more

Josh, thanks a lot. I restarted the cluster. It worked. Probably the issue was happening for slow start. ... View more

Hi Josh, Below is the log of master3.corp.mirrorplus.com- 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=:/usr/hdp/2.4.0.0-169/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.4.0.0-169/hadoop/lib/native 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-327.3.1.el7.x86_64 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=hbase 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/home/hbase 2016-04-04 05:47:12,159 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/hbase 2016-04-04 05:47:12,160 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 sessionTimeout=90000 watcher=master:160000x0, quorum=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181, baseZNode=/hbase-unsecure 2016-04-04 05:47:12,185 INFO [main-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server master1.corp.mirrorplus.com/10.1.1.94:2181. Will not attempt to authenticate using SASL (unknown error) 2016-04-04 05:47:12,192 INFO [main-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Socket connection established to master1.corp.mirrorplus.com/10.1.1.94:2181, initiating session 2016-04-04 05:47:12,201 INFO [main-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server master1.corp.mirrorplus.com/10.1.1.94:2181, sessionid = 0x153e0a3fe360007, negotiated timeout = 40000 2016-04-04 05:47:12,247 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2016-04-04 05:47:12,247 INFO [RpcServer.listener,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: starting 2016-04-04 05:47:12,304 INFO [main] mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2016-04-04 05:47:12,307 INFO [main] http.HttpRequestLog: Http request log for http.requests.master is not defined 2016-04-04 05:47:12,317 INFO [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2016-04-04 05:47:12,320 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2016-04-04 05:47:12,320 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2016-04-04 05:47:12,320 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2016-04-04 05:47:12,335 INFO [main] http.HttpServer: Jetty bound to port 16010 2016-04-04 05:47:12,335 INFO [main] mortbay.log: jetty-6.1.26.hwx 2016-04-04 05:47:12,693 INFO [main] mortbay.log: Started SelectChannelConnector@0.0.0.0:16010 2016-04-04 05:47:12,696 INFO [main] master.HMaster: hbase.rootdir=hdfs://aimprodcluster/apps/hbase/data1, hbase.cluster.distributed=true 2016-04-04 05:47:12,707 INFO [main] master.HMaster: Adding backup master ZNode /hbase-unsecure/backup-masters/master3.corp.mirrorplus.com,16000,1459763231138 2016-04-04 05:47:12,807 INFO [master3:16000.activeMasterManager] master.ActiveMasterManager: Another master is the active master, master2.corp.mirrorplus.com,16000,1459763227559; waiting to become the next active master 2016-04-04 05:47:12,827 INFO [master/master3.corp.mirrorplus.com/10.1.10.20:16000] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x79c1e5c2 connecting to ZooKeeper ensemble=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 2016-04-04 05:47:12,827 INFO [master/master3.corp.mirrorplus.com/10.1.10.20:16000] zookeeper.ZooKeeper: Initiating client connection, connectString=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 sessionTimeout=90000 watcher=hconnection-0x79c1e5c20x0, quorum=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181, baseZNode=/hbase-unsecure 2016-04-04 05:47:12,834 INFO [master/master3.corp.mirrorplus.com/10.1.10.20:16000-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server master1.corp.mirrorplus.com/10.1.1.94:2181. Will not attempt to authenticate using SASL (unknown error) 2016-04-04 05:47:12,835 INFO [master/master3.corp.mirrorplus.com/10.1.10.20:16000-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Socket connection established to master1.corp.mirrorplus.com/10.1.1.94:2181, initiating session 2016-04-04 05:47:12,839 INFO [master/master3.corp.mirrorplus.com/10.1.10.20:16000-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server master1.corp.mirrorplus.com/10.1.1.94:2181, sessionid = 0x153e0a3fe360008, negotiated timeout = 40000 2016-04-04 05:47:12,864 INFO [master/master3.corp.mirrorplus.com/10.1.10.20:16000] regionserver.HRegionServer: ClusterId : a330e3a3-4f5c-402c-b934-013315c0b547 ... View more

Hi Josh, I changed standby and active masters, below is the log of master2.corp.mirrorplus.com- 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=:/usr/hdp/2.4.0.0-169/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.4.0.0-169/hadoop/lib/native 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-327.3.1.el7.x86_64 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=hbase 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/home/hbase 2016-04-04 05:47:08,529 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/hbase 2016-04-04 05:47:08,530 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 sessionTimeout=90000 watcher=master:160000x0, quorum=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181, baseZNode=/hbase-unsecure 2016-04-04 05:47:08,575 INFO [main-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server master1.corp.mirrorplus.com/10.1.1.94:2181. Will not attempt to authenticate using SASL (unknown error) 2016-04-04 05:47:08,581 INFO [main-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Socket connection established to master1.corp.mirrorplus.com/10.1.1.94:2181, initiating session 2016-04-04 05:47:08,589 INFO [main-SendThread(master1.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server master1.corp.mirrorplus.com/10.1.1.94:2181, sessionid = 0x153e0a3fe360005, negotiated timeout = 40000 2016-04-04 05:47:08,648 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2016-04-04 05:47:08,648 INFO [RpcServer.listener,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: starting 2016-04-04 05:47:08,710 INFO [main] mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2016-04-04 05:47:08,716 INFO [main] http.HttpRequestLog: Http request log for http.requests.master is not defined 2016-04-04 05:47:08,730 INFO [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2016-04-04 05:47:08,733 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2016-04-04 05:47:08,733 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2016-04-04 05:47:08,733 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2016-04-04 05:47:08,753 INFO [main] http.HttpServer: Jetty bound to port 16010 2016-04-04 05:47:08,753 INFO [main] mortbay.log: jetty-6.1.26.hwx 2016-04-04 05:47:09,146 INFO [main] mortbay.log: Started SelectChannelConnector@0.0.0.0:16010 2016-04-04 05:47:09,151 INFO [main] master.HMaster: hbase.rootdir=hdfs://aimprodcluster/apps/hbase/data1, hbase.cluster.distributed=true 2016-04-04 05:47:09,167 INFO [main] master.HMaster: Adding backup master ZNode /hbase-unsecure/backup-masters/master2.corp.mirrorplus.com,16000,1459763227559 2016-04-04 05:47:09,270 INFO [master2:16000.activeMasterManager] master.ActiveMasterManager: Another master is the active master, master3.corp.mirrorplus.com,16000,1459762844022; waiting to become the next active master 2016-04-04 05:47:09,307 INFO [master/master2.corp.mirrorplus.com/10.1.1.95:16000] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x7063e847 connecting to ZooKeeper ensemble=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 2016-04-04 05:47:09,307 INFO [master/master2.corp.mirrorplus.com/10.1.1.95:16000] zookeeper.ZooKeeper: Initiating client connection, connectString=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 sessionTimeout=90000 watcher=hconnection-0x7063e8470x0, quorum=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181, baseZNode=/hbase-unsecure 2016-04-04 05:47:09,308 INFO [master/master2.corp.mirrorplus.com/10.1.1.95:16000-SendThread(master2.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server master2.corp.mirrorplus.com/10.1.1.95:2181. Will not attempt to authenticate using SASL (unknown error) 2016-04-04 05:47:09,308 INFO [master/master2.corp.mirrorplus.com/10.1.1.95:16000-SendThread(master2.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Socket connection established to master2.corp.mirrorplus.com/10.1.1.95:2181, initiating session 2016-04-04 05:47:09,314 INFO [master/master2.corp.mirrorplus.com/10.1.1.95:16000-SendThread(master2.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server master2.corp.mirrorplus.com/10.1.1.95:2181, sessionid = 0x253e0a3fe3e0008, negotiated timeout = 40000 2016-04-04 05:47:09,338 INFO [master/master2.corp.mirrorplus.com/10.1.1.95:16000] regionserver.HRegionServer: ClusterId : 562355a3-3569-4d2e-bf16-efaa82632c96 2016-04-04 05:47:09,437 INFO [master2:16000.activeMasterManager] master.ActiveMasterManager: Deleting ZNode for /hbase-unsecure/backup-masters/master2.corp.mirrorplus.com,16000,1459763227559 from backup master directory 2016-04-04 05:47:09,444 INFO [master2:16000.activeMasterManager] master.ActiveMasterManager: Registered Active Master=master2.corp.mirrorplus.com,16000,1459763227559 2016-04-04 05:47:09,833 INFO [master2:16000.activeMasterManager] util.FSUtils: Created version file at hdfs://aimprodcluster/apps/hbase/data1 with version=8 2016-04-04 05:47:09,934 INFO [master2:16000.activeMasterManager] master.MasterFileSystem: BOOTSTRAP: creating hbase:meta region 2016-04-04 05:47:09,937 INFO [master2:16000.activeMasterManager] regionserver.HRegion: creating HRegion hbase:meta HTD == 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, {NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '10', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'false', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} RootDir = hdfs://aimprodcluster/apps/hbase/data1 Table name == hbase:meta 2016-04-04 05:47:10,010 INFO [master2:16000.activeMasterManager] wal.WALFactory: Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.DefaultWALProvider 2016-04-04 05:47:10,035 INFO [master2:16000.activeMasterManager] wal.FSHLog: WAL configuration: blocksize=128 MB, rollsize=121.60 MB, prefix=hregion-44984721.default, suffix=, logDir=hdfs://aimprodcluster/apps/hbase/data1/WALs/hregion-44984721, archiveDir=hdfs://aimprodcluster/apps/hbase/data1/oldWALs 2016-04-04 05:47:10,085 INFO [master2:16000.activeMasterManager] wal.FSHLog: Slow sync cost: 33 ms, current pipeline: [] 2016-04-04 05:47:10,087 INFO [master2:16000.activeMasterManager] wal.FSHLog: New WAL /apps/hbase/data1/WALs/hregion-44984721/hregion-44984721.default.1459763230035 2016-04-04 05:47:10,182 INFO [StoreOpener-1588230740-1] hfile.CacheConfig: Allocating LruBlockCache size=5.99 GB, blockSize=64 KB ... View more

below is hbase master log- 2016-04-04 05:47:05,382 INFO [main] zookeeper.ZooKeeper: Client environment:java.class.path=/usr/hdp/current/hbase-master/conf:/usr/jdk64/jdk1.8.0_60/lib/tools.jar:/usr/hdp/current/hbase-master/bin/..:/usr/hdp/current/hbase-master/bin/../lib/activation-1.1.jar:/usr/hdp/current/hbase-master/bin/../lib/aopalliance-1.0.jar:/usr/hdp/current/hbase-master/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hbase-master/bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hbase-master/bin/../lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hbase-master/bin/../lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hbase-master/bin/../lib/asm-3.1.jar:/usr/hdp/current/hbase-master/bin/../lib/avro-1.7.4.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-cli-1.2.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-codec-1.9.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-collections-3.2.2.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-compress-1.4.1.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-configuration-1.6.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-digester-1.8.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-el-1.0.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-httpclient-3.1.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-io-2.4.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-lang-2.6.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-logging-1.2.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-math-2.2.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-math3-3.1.1.jar:/usr/hdp/current/hbase-master/bin/../lib/commons-net-3.1.jar:/usr/hdp/current/hbase-master/bin/../lib/curator-client-2.7.1.jar:/usr/hdp/current/hbase-master/bin/../lib/curator-framework-2.7.1.jar:/usr/hdp/current/hbase-master/bin/../lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hbase-master/bin/../lib/disruptor-3.3.0.jar:/usr/hdp/current/hbase-master/bin/../lib/findbugs-annotations-1.3.9-1.jar:/usr/hdp/current/hbase-master/bin/../lib/gson-2.2.4.jar:/usr/hdp/current/hbase-master/bin/../lib/guava-12.0.1.jar:/usr/hdp/current/hbase-master/bin/../lib/guice-3.0.jar:/usr/hdp/current/hbase-master/bin/../lib/guice-servlet-3.0.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-annotations-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-annotations-1.1.2.2.4.0.0-169-tests.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-annotations.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-client-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-client.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-common-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-common-1.1.2.2.4.0.0-169-tests.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-common.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-examples-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-examples.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-hadoop2-compat-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-hadoop2-compat.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-hadoop-compat.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-it-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-it-1.1.2.2.4.0.0-169-tests.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-it.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-prefix-tree-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-prefix-tree.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-procedure-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-procedure.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-protocol-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-protocol.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-resource-bundle-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-resource-bundle.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-rest-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-rest.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-server-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-server-1.1.2.2.4.0.0-169-tests.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-server.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-shell-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-shell.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-thrift-1.1.2.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/hbase-thrift.jar:/usr/hdp/current/hbase-master/bin/../lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hbase-master/bin/../lib/httpclient-4.2.5.jar:/usr/hdp/current/hbase-master/bin/../lib/httpcore-4.2.5.jar:/usr/hdp/current/hbase-master/bin/../lib/jackson-core-2.2.3.jar:/usr/hdp/current/hbase-master/bin/../lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hbase-master/bin/../lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hbase-master/bin/../lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hbase-master/bin/../lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hbase-master/bin/../lib/jamon-runtime-2.3.1.jar:/usr/hdp/current/hbase-master/bin/../lib/jasper-compiler-5.5.23.jar:/usr/hdp/current/hbase-master/bin/../lib/jasper-runtime-5.5.23.jar:/usr/hdp/current/hbase-master/bin/../lib/javax.inject-1.jar:/usr/hdp/current/hbase-master/bin/../lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hbase-master/bin/../lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hbase-master/bin/../lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hbase-master/bin/../lib/jcodings-1.0.8.jar:/usr/hdp/current/hbase-master/bin/../lib/jersey-client-1.9.jar:/usr/hdp/current/hbase-master/bin/../lib/jersey-core-1.9.jar:/usr/hdp/current/hbase-master/bin/../lib/jersey-guice-1.9.jar:/usr/hdp/current/hbase-master/bin/../lib/jersey-json-1.9.jar:/usr/hdp/current/hbase-master/bin/../lib/jersey-server-1.9.jar:/usr/hdp/current/hbase-master/bin/../lib/jets3t-0.9.0.jar:/usr/hdp/current/hbase-master/bin/../lib/jettison-1.3.3.jar:/usr/hdp/current/hbase-master/bin/../lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hbase-master/bin/../lib/jetty-sslengine-6.1.26.hwx.jar:/usr/hdp/current/hbase-master/bin/../lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hbase-master/bin/../lib/joni-2.1.2.jar:/usr/hdp/current/hbase-master/bin/../lib/jruby-complete-1.6.8.jar:/usr/hdp/current/hbase-master/bin/../lib/jsch-0.1.42.jar:/usr/hdp/current/hbase-master/bin/../lib/jsp-2.1-6.1.14.jar:/usr/hdp/current/hbase-master/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/hdp/current/hbase-master/bin/../lib/jsr305-1.3.9.jar:/usr/hdp/current/hbase-master/bin/../lib/junit-4.11.jar:/usr/hdp/current/hbase-master/bin/../lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hbase-master/bin/../lib/libthrift-0.9.0.jar:/usr/hdp/current/hbase-master/bin/../lib/log4j-1.2.17.jar:/usr/hdp/current/hbase-master/bin/../lib/metrics-core-2.2.0.jar:/usr/hdp/current/hbase-master/bin/../lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hbase-master/bin/../lib/netty-3.2.4.Final.jar:/usr/hdp/current/hbase-master/bin/../lib/netty-all-4.0.23.Final.jar:/usr/hdp/current/hbase-master/bin/../lib/ojdbc6.jar:/usr/hdp/current/hbase-master/bin/../lib/okhttp-2.4.0.jar:/usr/hdp/current/hbase-master/bin/../lib/okio-1.4.0.jar:/usr/hdp/current/hbase-master/bin/../lib/paranamer-2.3.jar:/usr/hdp/current/hbase-master/bin/../lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hbase-master/bin/../lib/ranger-hbase-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/current/hbase-master/bin/../lib/servlet-api-2.5-6.1.14.jar:/usr/hdp/current/hbase-master/bin/../lib/servlet-api-2.5.jar:/usr/hdp/current/hbase-master/bin/../lib/slf4j-api-1.7.7.jar:/usr/hdp/current/hbase-master/bin/../lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hbase-master/bin/../lib/spymemcached-2.11.6.jar:/usr/hdp/current/hbase-master/bin/../lib/xercesImpl-2.9.1.jar:/usr/hdp/current/hbase-master/bin/../lib/xml-apis-1.3.04.jar:/usr/hdp/current/hbase-master/bin/../lib/xmlenc-0.52.jar:/usr/hdp/current/hbase-master/bin/../lib/xz-1.0.jar:/usr/hdp/current/hbase-master/bin/../lib/zookeeper.jar:/usr/hdp/2.4.0.0-169/hadoop/conf:/usr/hdp/2.4.0.0-169/hadoop/lib/jersey-json-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/spark-yarn-shuffle.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ojdbc6.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-databind-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ranger-hdfs-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ranger-yarn-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/activation-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jersey-server-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-xc-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jets3t-0.9.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jettison-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/api-util-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/asm-3.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/avro-1.7.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jetty-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/aws-java-sdk-1.7.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/azure-storage-2.2.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/paranamer-2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-cli-1.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/netty-3.6.2.Final.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-codec-1.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jaxb-api-2.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-collections-3.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/protobuf-java-2.5.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-compress-1.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-configuration-1.6.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-digester-1.8.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/slf4j-api-1.7.10.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-httpclient-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-io-2.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-lang-2.6.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-logging-1.1.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-math3-3.1.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-net-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/stax-api-1.0-2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/curator-client-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/xmlenc-0.52.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/curator-framework-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/xz-1.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/curator-recipes-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/gson-2.2.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/guava-11.0.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/zookeeper-3.4.6.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/hamcrest-core-1.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/mockito-all-1.8.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/httpclient-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/httpcore-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jersey-core-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-annotations-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-core-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jsch-0.1.42.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jsp-api-2.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jsr305-3.0.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/junit-4.11.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/log4j-1.2.17.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-annotations-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-annotations.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-auth.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-aws-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-aws.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-azure-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-azure.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-common-2.7.1.2.4.0.0-169-tests.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-common-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-common-tests.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-common.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-nfs-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/.//hadoop-nfs.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/./:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/asm-3.2.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/jetty-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/.//hadoop-hdfs-2.7.1.2.4.0.0-169-tests.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/.//hadoop-hdfs-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/.//hadoop-hdfs-nfs-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/.//hadoop-hdfs-nfs.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/.//hadoop-hdfs-tests.jar:/usr/hdp/2.4.0.0-169/hadoop-hdfs/.//hadoop-hdfs.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/activation-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jets3t-0.9.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jettison-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/api-util-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/asm-3.2.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/avro-1.7.4.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/objenesis-2.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-beanutils-1.7.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/paranamer-2.3.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-configuration-1.6.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-digester-1.8.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-httpclient-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-io-2.4.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/snappy-java-1.0.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-math3-3.1.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/commons-net-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/xmlenc-0.52.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/xz-1.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/curator-framework-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/zookeeper-3.4.6.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/curator-recipes-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/fst-2.24.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/gson-2.2.4.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/guava-11.0.2.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/guice-3.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/zookeeper-3.4.6.2.4.0.0-169-tests.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/httpclient-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/httpcore-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jackson-annotations-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jackson-core-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jackson-databind-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/java-xmlbuilder-0.4.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/javax.inject-1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jetty-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jsch-0.1.42.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jsp-api-2.1.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-api-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-client-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-client.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-common-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-registry-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-registry.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-common-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-tests-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-timeline-plugins-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-timeline-plugins.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/asm-3.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/guice-3.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/junit-4.11.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/lib/xz-1.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jaxb-api-2.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//activation-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-sls-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//apacheds-i18n-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-app-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-sls.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//api-asn1-api-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hamcrest-core-1.3.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//api-util-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//log4j-1.2.17.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//asm-3.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jaxb-impl-2.2.3-1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//avro-1.7.4.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-streaming.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-beanutils-1.7.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-beanutils-core-1.8.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jersey-core-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-cli-1.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//httpcore-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-codec-1.4.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-tests.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-collections-3.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-compress-1.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-configuration-1.6.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//httpclient-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-digester-1.8.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jackson-core-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-httpclient-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jersey-json-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-io-2.4.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jersey-server-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-lang-2.6.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jackson-core-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-lang3-3.3.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jackson-jaxrs-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-logging-1.1.3.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//java-xmlbuilder-0.4.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-math3-3.1.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jets3t-0.9.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//commons-net-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jackson-mapper-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//curator-client-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jackson-xc-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//curator-framework-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//mockito-all-1.8.5.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//curator-recipes-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jettison-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//gson-2.2.4.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jetty-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//guava-11.0.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-ant-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jetty-util-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-ant.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-archives-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//joda-time-2.9.2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-archives.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jsch-0.1.42.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-auth.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-common-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-datajoin-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jsp-api-2.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-datajoin.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-rumen.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-distcp-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//jsr305-3.0.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-distcp.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-openstack-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-extras-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//junit-4.11.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-extras.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-openstack.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//xz-1.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-gridmix-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//metrics-core-3.0.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-gridmix.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-core-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-rumen-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-streaming-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.7.1.2.4.0.0-169-tests.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//hadoop-mapreduce-examples-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//netty-3.6.2.Final.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//paranamer-2.3.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//protobuf-java-2.5.0.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//snappy-java-1.0.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//stax-api-1.0-2.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//xmlenc-0.52.jar:/usr/hdp/2.4.0.0-169/hadoop-mapreduce/.//zookeeper-3.4.6.2.4.0.0-169.jar::mysql-connector-java.jar:/usr/hdp/2.4.0.0-169/hadoop/conf:/usr/hdp/2.4.0.0-169/hadoop/hadoop-annotations-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-annotations.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-auth-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-auth.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-aws-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-aws.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-azure-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-azure.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-common-2.7.1.2.4.0.0-169-tests.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-common-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-common-tests.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-common.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-nfs-2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/hadoop-nfs.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jersey-json-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/spark-yarn-shuffle.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ojdbc6.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-databind-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ranger-hdfs-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ranger-plugin-classloader-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/ranger-yarn-plugin-shim-0.5.0.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/activation-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jersey-server-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-xc-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jets3t-0.9.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jettison-1.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/api-util-1.0.0-M20.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/asm-3.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/avro-1.7.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jetty-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/aws-java-sdk-1.7.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/azure-storage-2.2.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/paranamer-2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-cli-1.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/netty-3.6.2.Final.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-codec-1.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jaxb-api-2.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-collections-3.2.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/protobuf-java-2.5.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-compress-1.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-configuration-1.6.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/servlet-api-2.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-digester-1.8.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/slf4j-api-1.7.10.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-httpclient-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-io-2.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-lang-2.6.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-logging-1.1.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-math3-3.1.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/commons-net-3.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/stax-api-1.0-2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/curator-client-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/xmlenc-0.52.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/curator-framework-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/xz-1.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/curator-recipes-2.7.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/gson-2.2.4.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/guava-11.0.2.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/zookeeper-3.4.6.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/hamcrest-core-1.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/mockito-all-1.8.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/httpclient-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/httpcore-4.2.5.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jersey-core-1.9.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-annotations-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-core-2.2.3.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jsch-0.1.42.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jsp-api-2.1.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/jsr305-3.0.0.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/junit-4.11.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/log4j-1.2.17.jar:/usr/hdp/2.4.0.0-169/hadoop/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/zookeeper-3.4.6.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/zookeeper/zookeeper.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/ant-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/ant-launcher-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/backport-util-concurrent-3.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/classworlds-1.1-alpha-2.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/commons-io-2.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/commons-logging-1.1.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/httpclient-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/httpcore-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/jsoup-1.7.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/log4j-1.2.16.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-ant-tasks-2.1.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-artifact-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-artifact-manager-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-error-diagnostics-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-model-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-plugin-registry-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-profile-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-project-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-repository-metadata-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-settings-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/nekohtml-1.9.6.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/netty-3.7.0.Final.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-container-default-1.0-alpha-9-stable-1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-interpolation-1.11.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-utils-3.0.8.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/slf4j-api-1.6.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/slf4j-log4j12-1.6.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-file-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-lightweight-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-shared-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-shared4-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-provider-api-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/xercesMinimal-1.9.6.2.jar: 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=:/usr/hdp/2.4.0.0-169/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.4.0.0-169/hadoop/lib/native 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-327.3.1.el7.x86_64 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=hbase 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/home/hbase 2016-04-04 05:47:05,384 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/hbase 2016-04-04 05:47:05,385 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 sessionTimeout=90000 watcher=master:160000x0, quorum=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181, baseZNode=/hbase-unsecure 2016-04-04 05:47:05,403 INFO [main-SendThread(master2.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server master2.corp.mirrorplus.com/10.1.1.95:2181. Will not attempt to authenticate using SASL (unknown error) 2016-04-04 05:47:05,408 INFO [main-SendThread(master2.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Socket connection established to master2.corp.mirrorplus.com/10.1.1.95:2181, initiating session 2016-04-04 05:47:05,466 INFO [main-SendThread(master2.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server master2.corp.mirrorplus.com/10.1.1.95:2181, sessionid = 0x253e0a3fe3e0007, negotiated timeout = 40000 2016-04-04 05:47:05,551 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2016-04-04 05:47:05,551 INFO [RpcServer.listener,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: starting 2016-04-04 05:47:05,612 INFO [main] mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2016-04-04 05:47:05,617 INFO [main] http.HttpRequestLog: Http request log for http.requests.master is not defined 2016-04-04 05:47:05,631 INFO [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2016-04-04 05:47:05,633 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2016-04-04 05:47:05,633 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2016-04-04 05:47:05,633 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2016-04-04 05:47:05,654 INFO [main] http.HttpServer: Jetty bound to port 16010 2016-04-04 05:47:05,654 INFO [main] mortbay.log: jetty-6.1.26.hwx 2016-04-04 05:47:06,186 INFO [main] mortbay.log: Started SelectChannelConnector@0.0.0.0:16010 2016-04-04 05:47:06,191 INFO [main] master.HMaster: hbase.rootdir=hdfs://aimprodcluster/apps/hbase/data1, hbase.cluster.distributed=true 2016-04-04 05:47:06,204 INFO [main] master.HMaster: Adding backup master ZNode /hbase-unsecure/backup-masters/master1.corp.mirrorplus.com,16000,1459763224469 2016-04-04 05:47:06,289 INFO [master1:16000.activeMasterManager] master.ActiveMasterManager: Another master is the active master, master3.corp.mirrorplus.com,16000,1459762844022; waiting to become the next active master 2016-04-04 05:47:06,336 INFO [master/master1.corp.mirrorplus.com/10.1.1.94:16000] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xd8b2c68 connecting to ZooKeeper ensemble=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 2016-04-04 05:47:06,336 INFO [master/master1.corp.mirrorplus.com/10.1.1.94:16000] zookeeper.ZooKeeper: Initiating client connection, connectString=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181 sessionTimeout=90000 watcher=hconnection-0xd8b2c680x0, quorum=master3.corp.mirrorplus.com:2181,master2.corp.mirrorplus.com:2181,master1.corp.mirrorplus.com:2181, baseZNode=/hbase-unsecure 2016-04-04 05:47:06,337 INFO [master/master1.corp.mirrorplus.com/10.1.1.94:16000-SendThread(master3.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server master3.corp.mirrorplus.com/10.1.10.20:2181. Will not attempt to authenticate using SASL (unknown error) 2016-04-04 05:47:06,338 INFO [master/master1.corp.mirrorplus.com/10.1.1.94:16000-SendThread(master3.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Socket connection established to master3.corp.mirrorplus.com/10.1.10.20:2181, initiating session 2016-04-04 05:47:06,346 INFO [master/master1.corp.mirrorplus.com/10.1.1.94:16000-SendThread(master3.corp.mirrorplus.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server master3.corp.mirrorplus.com/10.1.10.20:2181, sessionid = 0x353e0a406670007, negotiated timeout = 40000 2016-04-04 05:47:06,373 INFO [master/master1.corp.mirrorplus.com/10.1.1.94:16000] regionserver.HRegionServer: ClusterId : 562355a3-3569-4d2e-bf16-efaa82632c96 2016-04-04 05:47:09,438 INFO [master1:16000.activeMasterManager] master.ActiveMasterManager: Another master is the active master, master2.corp.mirrorplus.com,16000,1459763227559; waiting to become the next active master ... View more

Hi @Chris Nauroth, Thanks for the solution, It worked. I increased disk space, turned off hdfs safemode and started regionserver. It worked. Thanks, Raja Ray ... View more