Hi every one,
i have a 5 node cdh cluster.in my cluster i am observing that node managers are restarting continuously.
i am not sure what is going on i attaching the stdout and stderr and roll log.
can you please help me
Stderr
+ exec /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop-yarn/bin/yarn nodemanager
Aug 02, 2018 11:30:32 AM com.google.inject.servlet.InternalServletModule$BackwardsCompatibleServletContextProvider get
WARNING: You are attempting to use a deprecated API (specifically, attempting to @Inject ServletContext inside an eagerly created singleton. While we allow this for backwards compatibility, be warned that this MAY have unexpected behavior if you have more than one injector (with ServletModule) running in the same JVM. Please consult the Guice documentation at http://code.google.com/p/google-guice/wiki/Servlets for more information.
Aug 02, 2018 11:30:32 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class
Aug 02, 2018 11:30:32 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Aug 02, 2018 11:30:32 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class
Aug 02, 2018 11:30:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Aug 02, 2018 11:30:32 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Aug 02, 2018 11:30:32 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Aug 02, 2018 11:30:33 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton"
role log
11:48:42.105 AM INFO ContainerManagerImpl
Start request for container_1533205969497_0410_01_000001 by user dr.who
11:48:42.105 AM INFO ContainerManagerImpl
Creating a new application reference for app application_1533205969497_0410
11:48:42.105 AM INFO Application
Application application_1533205969497_0410 transitioned from NEW to INITING
11:48:42.106 AM INFO NMAuditLogger
USER=dr.who IP=172.31.24.227 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1533205969497_0410 CONTAINERID=container_1533205969497_0410_01_000001
11:48:42.108 AM INFO AppLogAggregatorImpl
rollingMonitorInterval is set as -1. The log rolling monitoring interval is disabled. The logs will be aggregated after this application is finished.
11:48:42.125 AM INFO Application
Adding container_1533205969497_0410_01_000001 to application application_1533205969497_0410
11:48:42.125 AM INFO Application
Application application_1533205969497_0410 transitioned from INITING to RUNNING
11:48:42.125 AM INFO Container
Container container_1533205969497_0410_01_000001 transitioned from NEW to LOCALIZED
11:48:42.125 AM INFO AuxServices
Got event CONTAINER_INIT for appId application_1533205969497_0410
11:48:42.125 AM INFO YarnShuffleService
Initializing container container_1533205969497_0410_01_000001
11:48:42.144 AM INFO Container
Container container_1533205969497_0410_01_000001 transitioned from LOCALIZED to RUNNING
11:48:42.147 AM INFO DefaultContainerExecutor
launchContainer: [bash, /data0/yarn/nm/usercache/dr.who/appcache/application_1533205969497_0410/container_1533205969497_0410_01_000001/default_container_executor.sh]
11:48:42.162 AM WARN DefaultContainerExecutor
Exit code from container container_1533205969497_0410_01_000001 is : 143
11:48:42.164 AM INFO Container
Container container_1533205969497_0410_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE
11:48:42.164 AM INFO ContainerLaunch
Cleaning up container container_1533205969497_0410_01_000001
11:48:42.181 AM INFO DefaultContainerExecutor
Deleting absolute path : /data0/yarn/nm/usercache/dr.who/appcache/application_1533205969497_0410/container_1533205969497_0410_01_000001
11:48:42.182 AM WARN NMAuditLogger
USER=dr.who OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1533205969497_0410 CONTAINERID=container_1533205969497_0410_01_000001
11:48:42.182 AM INFO Container
Container container_1533205969497_0410_01_000001 transitioned from EXITED_WITH_FAILURE to DONE
11:48:42.182 AM INFO Application
Removing container_1533205969497_0410_01_000001 from application application_1533205969497_0410
11:48:42.182 AM INFO AppLogAggregatorImpl
Considering container container_1533205969497_0410_01_000001 for log-aggregation
11:48:42.182 AM INFO AuxServices
Got event CONTAINER_STOP for appId application_1533205969497_0410
11:48:42.182 AM INFO YarnShuffleService
Stopping container container_1533205969497_0410_01_000001
11:48:43.185 AM INFO NodeStatusUpdaterImpl
Removed completed containers from NM context: [container_1533205969497_0410_01_000001]
