2018-07-07 11:10:47,926 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 11:10:47,929 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:43:48,601 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config. 2018-07-07 12:43:49,913 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:43:49,919 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:43:51,615 WARN launcher.RecoveredContainerLaunch (RecoveredContainerLaunch.java:call(106)) - Recovered container exited with a non-zero exit code 143 2018-07-07 12:43:51,757 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0003 CONTAINERID=container_e09_1530955519179_0003_01_000001 2018-07-07 12:43:51,962 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:43:51,963 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:43:52,253 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:43:52,582 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0184_01_000001 is : 143 2018-07-07 12:43:52,583 WARN containermanager.ContainerManagerImpl (ContainerManagerImpl.java:handle(1080)) - Event EventType: KILL_CONTAINER sent to absent container container_e09_1530955519179_0003_01_000001 2018-07-07 12:43:52,655 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0184 CONTAINERID=container_e09_1530955519179_0184_01_000001 2018-07-07 12:43:52,667 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:43:52,668 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:43:52,896 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0185_01_000001 is : 143 2018-07-07 12:43:52,964 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0185 CONTAINERID=container_e09_1530955519179_0185_01_000001 2018-07-07 12:43:53,663 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:43:53,663 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:43:54,632 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:43:54,632 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:43:55,632 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:43:55,633 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:43:56,661 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:43:56,662 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:00,084 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0184_02_000001 is : 143 2018-07-07 12:44:00,196 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0184 CONTAINERID=container_e09_1530955519179_0184_02_000001 2018-07-07 12:44:00,883 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0185_02_000001 is : 143 2018-07-07 12:44:00,995 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0185 CONTAINERID=container_e09_1530955519179_0185_02_000001 2018-07-07 12:44:01,755 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:01,759 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:02,754 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:02,737 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:06,916 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:06,916 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:07,100 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:07,101 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:07,142 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:07,143 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:07,268 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0190_01_000001 is : 143 2018-07-07 12:44:07,374 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0190 CONTAINERID=container_e09_1530955519179_0190_01_000001 2018-07-07 12:44:07,883 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:07,884 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:08,096 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0191_01_000001 is : 143 2018-07-07 12:44:08,158 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0191 CONTAINERID=container_e09_1530955519179_0191_01_000001 2018-07-07 12:44:08,536 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException 2018-07-07 12:44:08,822 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:08,823 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:08,891 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:08,891 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:09,159 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0190_02_000001 is : 143 2018-07-07 12:44:09,246 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0190 CONTAINERID=container_e09_1530955519179_0190_02_000001 2018-07-07 12:44:10,017 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_e09_1530955519179_0191_02_000001 is : 143 2018-07-07 12:44:10,071 WARN nodemanager.NMAuditLogger (NMAuditLogger.java:logFailure(150)) - USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1530955519179_0191 CONTAINERID=container_e09_1530955519179_0191_02_000001 2018-07-07 12:44:10,810 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:10,810 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:10,863 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:10,864 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:11,861 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:11,862 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:11,863 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:11,861 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:17,853 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:17,854 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:18,536 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException 2018-07-07 12:44:21,911 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:21,911 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:22,897 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:22,907 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:28,536 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException 2018-07-07 12:44:28,929 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:28,929 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:29,934 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:29,935 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:38,000 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:38,000 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:38,544 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException 2018-07-07 12:44:39,032 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:39,032 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:47,108 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:47,108 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:48,536 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException 2018-07-07 12:44:49,155 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:49,155 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2018-07-07 12:44:58,216 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:58,217 ERROR nodemanager.DeletionService (DeletionService.java:afterExecute(182)) - Exception during execution of task in DeletionService 2018-07-07 12:44:59,243 WARN logaggregation.LogAggregationService (LogAggregationService.java:verifyAndCreateRemoteLogDir(195)) - Remote Root Log Dir [/app-logs] already exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The cluster may have problems with multiple users. 2018-07-07 12:44:59,243 WARN logaggregation.AppLogAggregatorImpl (AppLogAggregatorImpl.java:(190)) - rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished.