Support Questions

Find answers, ask questions, and share your expertise

How to resolve this Error with ZooKeeper while trying to deploy our model Mass_service (metron) ?

avatar
Explorer

@John Martin@asubramanian@Rajkumar Singh@Gonçalo Cunha@ssivachandran

While try to register our model with Zookeeper We are facing the following issue please try to give some solution how to fix it?

Log Type: AppMaster.stderr

Log Upload Time: Wed Mar 14 13:41:53 +0000 2018

Log Length: 48702

18/03/14 13:13:46 INFO service.ApplicationMaster: Initializing ApplicationMaster
18/03/14 13:13:47 INFO service.ApplicationMaster: Application master for app, appId=10, clustertimestamp=1520441972994, attemptId=1
18/03/14 13:13:47 INFO service.ApplicationMaster: Starting ApplicationMaster
18/03/14 13:13:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/03/14 13:13:47 INFO yarn.YarnUtils: Executing with tokens:
18/03/14 13:13:47 INFO yarn.YarnUtils: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 10 cluster_timestamp: 1520441972994 } attemptId: 1 } keyId: -86216004)
18/03/14 13:13:49 INFO impl.TimelineClientImpl: Timeline service address: http://ip-172-31-15-70.ap-south-1.compute.internal:8188/ws/v1/timeline/
18/03/14 13:13:50 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
18/03/14 13:13:51 INFO client.RMProxy: Connecting to ResourceManager at ip-172-31-15-70.ap-south-1.compute.internal/172.31.15.70:8030
18/03/14 13:13:51 INFO impl.NMClientAsyncImpl: Upper bound of the thread pool size is 500
18/03/14 13:13:51 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
18/03/14 13:13:52 INFO service.ApplicationMaster: Max mem capabililty of resources in this cluster 5120
18/03/14 13:13:52 INFO service.ApplicationMaster: Max vcores capabililty of resources in this cluster 1
18/03/14 13:13:52 INFO imps.CuratorFrameworkImpl: Starting
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:host.name=ip-172-31-9-203.ap-south-1.compute.internal
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_112
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/jdk64/jdk1.8.0_112/jre
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:java.class.path=$CLASSPATH:./AppMaster.jar:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/azure-data-lake-store-sdk-2.1.4.jar:/usr/hdp/current/hadoop-client/hadoop-annotations-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-annotations.jar:/usr/hdp/current/hadoop-client/hadoop-auth-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-auth.jar:/usr/hdp/current/hadoop-client/hadoop-aws-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-aws.jar:/usr/hdp/current/hadoop-client/hadoop-azure-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-azure-datalake-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-azure-datalake.jar:/usr/hdp/current/hadoop-client/hadoop-azure.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.3.2.6.1.0-129-tests.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-common-tests.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hadoop-client/hadoop-nfs-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-nfs.jar:/usr/hdp/current/hadoop-client/lib/ojdbc6.jar:/usr/hdp/current/hadoop-client/lib/nimbus-jose-jwt-3.9.jar:/usr/hdp/current/hadoop-client/lib/ranger-hdfs-plugin-shim-0.7.0.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/ranger-plugin-classloader-0.7.0.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/ranger-yarn-plugin-shim-0.7.0.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/jetty-sslengine-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-core-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-kms-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/joda-time-2.9.4.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-s3-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/azure-keyvault-core-0.8.0.jar:/usr/hdp/current/hadoop-client/lib/jsch-0.1.54.jar:/usr/hdp/current/hadoop-client/lib/azure-storage-4.2.0.jar:/usr/hdp/current/hadoop-client/lib/json-smart-1.1.1.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-client/lib/junit-4.11.jar:/usr/hdp/current/hadoop-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-client/lib/mockito-all-1.8.5.jar:/usr/hdp/current/hadoop-client/lib/commons-lang3-3.4.jar:/usr/hdp/current/hadoop-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/slf4j-api-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-client/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/hamcrest-core-1.3.jar:/usr/hdp/current/hadoop-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-client/lib/httpclient-4.5.2.jar:/usr/hdp/current/hadoop-client/lib/httpcore-4.4.4.jar:/usr/hdp/current/hadoop-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-client/lib/jcip-annotations-1.0.jar:/usr/hdp/current/hadoop-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-client/lib/zookeeper-3.4.6.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.3.2.6.1.0-129-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs.jar:/usr/hdp/current/hadoop-hdfs-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-hdfs-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-all-4.0.23.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okhttp-2.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okio-1.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xercesImpl-2.9.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xml-apis-1.3.04.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-pluginstorage-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy.jar:/usr/hdp/current/hadoop-yarn-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/aopalliance-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsch-0.1.54.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-sslengine-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/azure-keyvault-core-0.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/azure-storage-4.2.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/json-smart-1.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/javassist-3.18.1-GA.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/javax.inject-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-guice-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/metrics-core-3.0.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang3-3.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/nimbus-jose-jwt-3.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/objenesis-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/fst-2.24.jar:/usr/hdp/current/hadoop-yarn-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-servlet-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-yarn-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpclient-4.5.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpcore-4.4.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jcip-annotations-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-client-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.6.1.0-129-tests.jar:./log4j.properties:./AppMaster.jar:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/azure-data-lake-store-sdk-2.1.4.jar:/usr/hdp/current/hadoop-client/hadoop-annotations-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-annotations.jar:/usr/hdp/current/hadoop-client/hadoop-auth-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-auth.jar:/usr/hdp/current/hadoop-client/hadoop-aws-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-aws.jar:/usr/hdp/current/hadoop-client/hadoop-azure-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-azure-datalake-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-azure-datalake.jar:/usr/hdp/current/hadoop-client/hadoop-azure.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.3.2.6.1.0-129-tests.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-common-tests.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hadoop-client/hadoop-nfs-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/hadoop-nfs.jar:/usr/hdp/current/hadoop-client/lib/ojdbc6.jar:/usr/hdp/current/hadoop-client/lib/nimbus-jose-jwt-3.9.jar:/usr/hdp/current/hadoop-client/lib/ranger-hdfs-plugin-shim-0.7.0.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/ranger-plugin-classloader-0.7.0.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/ranger-yarn-plugin-shim-0.7.0.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/jetty-sslengine-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-core-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-kms-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/joda-time-2.9.4.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-s3-1.10.6.jar:/usr/hdp/current/hadoop-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/azure-keyvault-core-0.8.0.jar:/usr/hdp/current/hadoop-client/lib/jsch-0.1.54.jar:/usr/hdp/current/hadoop-client/lib/azure-storage-4.2.0.jar:/usr/hdp/current/hadoop-client/lib/json-smart-1.1.1.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-client/lib/junit-4.11.jar:/usr/hdp/current/hadoop-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-client/lib/mockito-all-1.8.5.jar:/usr/hdp/current/hadoop-client/lib/commons-lang3-3.4.jar:/usr/hdp/current/hadoop-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/slf4j-api-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-client/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/hamcrest-core-1.3.jar:/usr/hdp/current/hadoop-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-client/lib/httpclient-4.5.2.jar:/usr/hdp/current/hadoop-client/lib/httpcore-4.4.4.jar:/usr/hdp/current/hadoop-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-client/lib/jcip-annotations-1.0.jar:/usr/hdp/current/hadoop-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-client/lib/zookeeper-3.4.6.2.6.1.0-129.jar:/usr/hdp/current/hadoop-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.3.2.6.1.0-129-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs.jar:/usr/hdp/current/hadoop-hdfs-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-hdfs-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-all-4.0.23.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okhttp-2.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okio-1.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xercesImpl-2.9.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xml-apis-1.3.04.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-pluginstorage-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy-2.7.3.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy.jar:/usr/hdp/current/hadoop-yarn-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/aopalliance-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsch-0.1.54.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-sslengine-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/azure-keyvault-core-0.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/azure-storage-4.2.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/json-smart-1.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/javassist-3.18.1-GA.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/javax.inject-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-guice-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/metrics-core-3.0.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang3-3.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/nimbus-jose-jwt-3.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/objenesis-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/fst-2.24.jar:/usr/hdp/current/hadoop-yarn-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-servlet-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-yarn-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpclient-4.5.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpcore-4.4.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.6.1.0-129.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jcip-annotations-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-client-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.6.1.0-129-tests.jar:./log4j.properties
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.0-693.el7.x86_64
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:user.name=yarn
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/yarn
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Client environment:user.dir=/data/hadoop/yarn/local/usercache/root/appcache/application_1520441972994_0010/container_e61_1520441972994_0010_01_000001
18/03/14 13:13:52 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=ip-172-31-2-188.ap-south-1.compute.internal:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@51bde877
18/03/14 13:13:52 INFO zookeeper.ClientCnxn: Opening socket connection to server ip-172-31-2-188.ap-south-1.compute.internal/172.31.2.188:2181. Will not attempt to authenticate using SASL (unknown error)
18/03/14 13:13:52 INFO zookeeper.ClientCnxn: Socket connection established to ip-172-31-2-188.ap-south-1.compute.internal/172.31.2.188:2181, initiating session
18/03/14 13:13:52 INFO zookeeper.ClientCnxn: Session establishment complete on server ip-172-31-2-188.ap-south-1.compute.internal/172.31.2.188:2181, sessionid = 0x261fa134dd20059, negotiated timeout = 60000
18/03/14 13:13:52 INFO state.ConnectionStateManager: State change: CONNECTED
18/03/14 13:13:52 INFO service.ApplicationMaster: Ready to accept requests...
18/03/14 13:13:52 INFO service.ApplicationMaster: [ADD]: Received request for model dga:1.0x1 containers of size 512M at path /tmp/models
18/03/14 13:13:54 INFO impl.AMRMClientImpl: Received new token for : ip-172-31-9-203.ap-south-1.compute.internal:45454
18/03/14 13:13:54 INFO callback.ContainerRequestListener: Got response from RM for container ask, allocatedCnt=1
18/03/14 13:13:54 INFO callback.ContainerRequestListener: Launching shell command on a new container., containerId=container_e61_1520441972994_0010_01_000002, containerNode=ip-172-31-9-203.ap-south-1.compute.internal:45454, containerNodeURI=ip-172-31-9-203.ap-south-1.compute.internal:8042, containerResourceMemory=1024, containerResourceVirtualCores=1
18/03/14 13:13:54 INFO service.ApplicationMaster: Found container id of 67070209294338
18/03/14 13:13:54 INFO callback.LaunchContainer: Setting up container launch container for containerid=container_e61_1520441972994_0010_01_000002
18/03/14 13:13:54 INFO callback.LaunchContainer: Local Directory Contents
18/03/14 13:13:54 INFO callback.LaunchContainer:   19723217 - AppMaster.jar
18/03/14 13:13:54 INFO callback.LaunchContainer:   673 - default_container_executor_session.sh
18/03/14 13:13:54 INFO callback.LaunchContainer:   16 - .default_container_executor.sh.crc
18/03/14 13:13:54 INFO callback.LaunchContainer:   16 - .default_container_executor_session.sh.crc
18/03/14 13:13:54 INFO callback.LaunchContainer:   12 - .container_tokens.crc
18/03/14 13:13:54 INFO callback.LaunchContainer:   40 - .launch_container.sh.crc
18/03/14 13:13:54 INFO callback.LaunchContainer:   727 - default_container_executor.sh
18/03/14 13:13:54 INFO callback.LaunchContainer:   4096 - tmp
18/03/14 13:13:54 INFO callback.LaunchContainer:   74 - container_tokens
18/03/14 13:13:54 INFO callback.LaunchContainer:   3993 - launch_container.sh
18/03/14 13:13:54 INFO callback.LaunchContainer: Localizing /tmp/models
18/03/14 13:13:54 INFO callback.LaunchContainer: Model payload: /tmp/models
18/03/14 13:13:54 INFO callback.LaunchContainer: AppJAR Location: hdfs://ip-172-31-15-70.ap-south-1.compute.internal:8020/user/root/MaaS/application_1520441972994_0010/AppMaster.jar
18/03/14 13:13:54 INFO callback.LaunchContainer: Localized dga.py -> LocatedFileStatus{path=hdfs://ip-172-31-15-70.ap-south-1.compute.internal:8020/tmp/models/dga.py; isDirectory=false; length=755; replication=3; blocksize=134217728; modification_time=1521033074997; access_time=1521033074427; owner=root; group=hdfs; permission=rw-r--r--; isSymlink=false}
18/03/14 13:13:54 INFO callback.LaunchContainer: Localized rest.sh -> LocatedFileStatus{path=hdfs://ip-172-31-15-70.ap-south-1.compute.internal:8020/tmp/models/rest.sh; isDirectory=false; length=25; replication=3; blocksize=134217728; modification_time=1521033075154; access_time=1521033075002; owner=root; group=hdfs; permission=rw-r--r--; isSymlink=false}
18/03/14 13:13:54 INFO callback.LaunchContainer: AppMaster.jar localized: scheme: "hdfs" host: "ip-172-31-15-70.ap-south-1.compute.internal" port: 8020 file: "/user/root/MaaS/application_1520441972994_0010/AppMaster.jar"
18/03/14 13:13:54 INFO callback.LaunchContainer: dga.py localized: scheme: "hdfs" host: "ip-172-31-15-70.ap-south-1.compute.internal" port: 8020 file: "/tmp/models/dga.py"
18/03/14 13:13:54 INFO callback.LaunchContainer: rest.sh localized: scheme: "hdfs" host: "ip-172-31-15-70.ap-south-1.compute.internal" port: 8020 file: "/tmp/models/rest.sh"
18/03/14 13:13:54 INFO callback.LaunchContainer: Executing container command: {{JAVA_HOME}}/bin/java org.apache.metron.maas.service.runner.Runner -ci 67070209294338 -zq ip-172-31-2-188.ap-south-1.compute.internal:2181 -zr /metron/maas/config -s rest.sh -n dga -hn ip-172-31-9-203.ap-south-1.compute.internal -v 1.0 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr
18/03/14 13:13:55 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_e61_1520441972994_0010_01_000002
18/03/14 13:13:55 INFO impl.ContainerManagementProtocolProxy: Opening proxy : ip-172-31-9-203.ap-south-1.compute.internal:45454
18/03/14 13:13:55 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_e61_1520441972994_0010_01_000002
18/03/14 13:13:55 INFO impl.ContainerManagementProtocolProxy: Opening proxy : ip-172-31-9-203.ap-south-1.compute.internal:45454
18/03/14 13:17:35 INFO service.ApplicationMaster: [ADD]: Received request for model dga:1.0x1 containers of size 512M at path /tmp/models
18/03/14 13:17:36 INFO impl.AMRMClientImpl: Received new token for : ip-172-31-2-188.ap-south-1.compute.internal:45454
18/03/14 13:17:36 INFO callback.ContainerRequestListener: Got response from RM for container ask, allocatedCnt=2
18/03/14 13:17:36 INFO callback.ContainerRequestListener: Launching shell command on a new container., containerId=container_e61_1520441972994_0010_01_000003, containerNode=ip-172-31-9-203.ap-south-1.compute.internal:45454, containerNodeURI=ip-172-31-9-203.ap-south-1.compute.internal:8042, containerResourceMemory=1024, containerResourceVirtualCores=1
18/03/14 13:17:36 INFO callback.ContainerRequestListener: Launching shell command on a new container., containerId=container_e61_1520441972994_0010_01_000004, containerNode=ip-172-31-2-188.ap-south-1.compute.internal:45454, containerNodeURI=ip-172-31-2-188.ap-south-1.compute.internal:8042, containerResourceMemory=1024, containerResourceVirtualCores=1
18/03/14 13:17:36 INFO service.ApplicationMaster: Found container id of 67070209294339
18/03/14 13:17:36 INFO callback.LaunchContainer: Setting up container launch container for containerid=container_e61_1520441972994_0010_01_000003
18/03/14 13:17:36 INFO callback.LaunchContainer: Local Directory Contents
18/03/14 13:17:36 INFO callback.LaunchContainer:   19723217 - AppMaster.jar
18/03/14 13:17:36 INFO callback.LaunchContainer:   673 - default_container_executor_session.sh
18/03/14 13:17:36 INFO callback.LaunchContainer:   16 - .default_container_executor.sh.crc
18/03/14 13:17:36 INFO callback.LaunchContainer:   16 - .default_container_executor_session.sh.crc
18/03/14 13:17:36 INFO callback.LaunchContainer:   12 - .container_tokens.crc
18/03/14 13:17:36 INFO callback.LaunchContainer:   40 - .launch_container.sh.crc
18/03/14 13:17:36 INFO callback.LaunchContainer:   727 - default_container_executor.sh
18/03/14 13:17:36 INFO callback.LaunchContainer:   4096 - tmp
18/03/14 13:17:36 INFO callback.LaunchContainer:   74 - container_tokens
18/03/14 13:17:36 INFO callback.LaunchContainer:   3993 - launch_container.sh
18/03/14 13:17:36 INFO callback.LaunchContainer: Localizing /tmp/models
18/03/14 13:17:36 INFO callback.LaunchContainer: Model payload: /tmp/models
18/03/14 13:17:36 INFO callback.LaunchContainer: AppJAR Location: hdfs://ip-172-31-15-70.ap-south-1.compute.internal:8020/user/root/MaaS/application_1520441972994_0010/AppMaster.jar
18/03/14 13:17:36 INFO callback.LaunchContainer: Localized dga.py -> LocatedFileStatus{path=hdfs://ip-172-31-15-70.ap-south-1.compute.internal:8020/tmp/models/dga.py; isDirectory=false; length=755; replication=3; blocksize=134217728; modification_time=1521033455192; access_time=1521033454457; owner=root; group=hdfs; permission=rw-r--r--; isSymlink=false}
18/03/14 13:17:36 INFO callback.LaunchContainer: Localized rest.sh -> LocatedFileStatus{path=hdfs://ip-172-31-15-70.ap-south-1.compute.internal:8020/tmp/models/rest.sh; isDirectory=false; length=25; replication=3; blocksize=134217728; modification_time=1521033455275; access_time=1521033455198; owner=root; group=hdfs; permission=rw-r--r--; isSymlink=false}
18/03/14 13:17:36 INFO callback.LaunchContainer: AppMaster.jar localized: scheme: "hdfs" host: "ip-172-31-15-70.ap-south-1.compute.internal" port: 8020 file: "/user/root/MaaS/application_1520441972994_0010/AppMaster.jar"
18/03/14 13:17:36 INFO callback.LaunchContainer: dga.py localized: scheme: "hdfs" host: "ip-172-31-15-70.ap-south-1.compute.internal" port: 8020 file: "/tmp/models/dga.py"
18/03/14 13:17:36 INFO callback.LaunchContainer: rest.sh localized: scheme: "hdfs" host: "ip-172-31-15-70.ap-south-1.compute.internal" port: 8020 file: "/tmp/models/rest.sh"
18/03/14 13:17:36 INFO callback.LaunchContainer: Executing container command: {{JAVA_HOME}}/bin/java org.apache.metron.maas.service.runner.Runner -ci 67070209294339 -zq ip-172-31-2-188.ap-south-1.compute.internal:2181 -zr /metron/maas/config -s rest.sh -n dga -hn ip-172-31-9-203.ap-south-1.compute.internal -v 1.0 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr
18/03/14 13:17:36 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_e61_1520441972994_0010_01_000003
18/03/14 13:17:36 INFO impl.ContainerManagementProtocolProxy: Opening proxy : ip-172-31-9-203.ap-south-1.compute.internal:45454
18/03/14 13:17:37 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_e61_1520441972994_0010_01_000003
18/03/14 13:17:37 INFO impl.ContainerManagementProtocolProxy: Opening proxy : ip-172-31-9-203.ap-south-1.compute.internal:45454
18/03/14 13:21:16 ERROR service.ApplicationMaster: Received a null request...
18/03/14 13:24:00 INFO callback.ContainerRequestListener: Got response from RM for container ask, completedCnt=1
18/03/14 13:24:00 INFO callback.ContainerRequestListener: Got container status for containerID=container_e61_1520441972994_0010_01_000002, state=COMPLETE, exitStatus=1, diagnostics=Exception from container-launch.
Container id: container_e61_1520441972994_0010_01_000002
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
	at org.apache.hadoop.util.Shell.runCommand(Shell.java:944)
	at org.apache.hadoop.util.Shell.run(Shell.java:848)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1142)
	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:317)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:83)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1

18/03/14 13:24:00 INFO callback.ContainerRequestListener: REMOVING CONTAINER container_e61_1520441972994_0010_01_000002
18/03/14 13:24:00 WARN discovery.ServiceDiscoverer: Unable to find registered model associated with container container_e61_1520441972994_0010_01_000002
18/03/14 13:24:00 ERROR discovery.ServiceDiscoverer: Unable to unregister container container_e61_1520441972994_0010_01_000002 due to: Unable.
java.lang.IllegalStateException: Unable.
	at org.apache.metron.maas.discovery.ServiceDiscoverer.unregisterByContainer(ServiceDiscoverer.java:209)
	at org.apache.metron.maas.service.callback.ContainerRequestListener.onContainersCompleted(ContainerRequestListener.java:121)
	at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:296)
18/03/14 13:27:41 INFO callback.ContainerRequestListener: Got response from RM for container ask, completedCnt=1
18/03/14 13:27:41 INFO callback.ContainerRequestListener: Got container status for containerID=container_e61_1520441972994_0010_01_000003, state=COMPLETE, exitStatus=1, diagnostics=Exception from container-launch.
Container id: container_e61_1520441972994_0010_01_000003
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
	at org.apache.hadoop.util.Shell.runCommand(Shell.java:944)
	at org.apache.hadoop.util.Shell.run(Shell.java:848)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1142)
	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:317)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:83)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1

18/03/14 13:27:41 INFO callback.ContainerRequestListener: REMOVING CONTAINER container_e61_1520441972994_0010_01_000003
18/03/14 13:27:41 WARN discovery.ServiceDiscoverer: Unable to find registered model associated with container container_e61_1520441972994_0010_01_000003
18/03/14 13:27:41 ERROR discovery.ServiceDiscoverer: Unable to unregister container container_e61_1520441972994_0010_01_000003 due to: Unable.
java.lang.IllegalStateException: Unable.
	at org.apache.metron.maas.discovery.ServiceDiscoverer.unregisterByContainer(ServiceDiscoverer.java:209)
	at org.apache.metron.maas.service.callback.ContainerRequestListener.onContainersCompleted(ContainerRequestListener.java:121)
	at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:296)
18/03/14 13:29:43 INFO callback.ContainerRequestListener: Got response from RM for container ask, completedCnt=1
18/03/14 13:29:43 INFO callback.ContainerRequestListener: Got container status for containerID=container_e61_1520441972994_0010_01_000004, state=COMPLETE, exitStatus=-100, diagnostics=Container expired since it was unused
18/03/14 13:29:43 INFO callback.ContainerRequestListener: REMOVING CONTAINER container_e61_1520441972994_0010_01_000004
18/03/14 13:29:43 WARN discovery.ServiceDiscoverer: Unable to find registered model associated with container container_e61_1520441972994_0010_01_000004
18/03/14 13:29:43 ERROR discovery.ServiceDiscoverer: Unable to unregister container container_e61_1520441972994_0010_01_000004 due to: Unable.
java.lang.IllegalStateException: Unable.
	at org.apache.metron.maas.discovery.ServiceDiscoverer.unregisterByContainer(ServiceDiscoverer.java:209)
	at org.apache.metron.maas.service.callback.ContainerRequestListener.onContainersCompleted(ContainerRequestListener.java:121)
	at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:296)
18/03/14 13:37:35 ERROR service.ApplicationMaster: Received a null request... 
18/03/14 13:38:53 ERROR service.ApplicationMaster: Received a null request...

3 REPLIES 3

avatar
Contributor

avatar
Explorer

@Maxim Dashenko which version of Metron you are using? ... Are you using HCP the latest one?

avatar
Contributor

I'm using metron 0.4.3, to get the exact error that causes the issue you need to check the container's logs that are located on one of yarn nodes on /hadoop/yarn/log/{application_id}/{containter_id}/stderr