Skip to main content
sanjaysubs
sanjaysubs
Gold ⭐️
December 31, 2022
Solved

Map Reduce job on YARN hangs in ACCEPTED state

I have a three node cluster Screen Shot 2022-12-30 at 7.12.26 PM.png

 

Using cores=8 memory=32GB disks=2 hbase=False
Profile: cores=8 memory=31744MB reserved=1GB usableMem=31GB disks=2
Num Container=4
Container Ram=7680MB
Used Ram=30GB
Unused Ram=1GB
yarn.scheduler.minimum-allocation-mb=7680
yarn.scheduler.maximum-allocation-mb=30720
yarn.nodemanager.resource.memory-mb=30720
mapreduce.map.memory.mb=3840
mapreduce.map.java.opts=-Xmx3072m
mapreduce.reduce.memory.mb=7680
mapreduce.reduce.java.opts=-Xmx6144m
yarn.app.mapreduce.am.resource.mb=3840
yarn.app.mapreduce.am.command-opts=-Xmx3072m
mapreduce.task.io.sort.mb=1024

 

sanjaysubs
sanjaysubsAnswer
Gold ⭐️
January 3, 2023

OK I got it. In the yarn-site.xml I set the following in the Safety Valve

yarn.nodemanager.disk-health-checker.enable = false

View in thread

9 replies

AsimShaikh
Gold ⭐️
AsimShaikh
Gold ⭐️
January 2, 2023

Hi @sanjaysubs 

Seems you don't have any healthy NM currently thats why you don't have any resources available to run any application.

Please fix issue with your Node Managers it should solve your problem.

 

 

Translate
sanjaysubs
sanjaysubsAuthor
Gold ⭐️
January 2, 2023

U r right. The errors are as follows 

 

[ /media/sanjay/hdd02/yarn/nm : Cannot create directory: /media/sanjay/hdd02/yarn/nm , /media/sanjay/hdd03/yarn/nm : Cannot create directory: /media/sanjay/hdd03/yarn/nm ]

 

I have tried changing permissions / owners of the folders (yarn:hadoop) and in ultimate desperation made the folders 777 as well just to see if the problem gets solved. No luck yet...Not sure how to solve this. Running out of ideas fast LOL

Translate
sanjaysubs
sanjaysubsAuthor
Gold ⭐️
January 2, 2023

2023-01-01 17:31:58,726 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 2/2 local-dirs have errors: [ /media/sanjay/hdd02/yarn/nm : Cannot create directory
: /media/sanjay/hdd02/yarn/nm, error mkdir of /media/sanjay/hdd02 failed , /media/sanjay/hdd03/yarn/nm : Cannot create directory : /media/sanjay/hdd03/yarn/nm, error mkdir of /media/sanjay/hdd03 failed ]

Translate
sanjaysubs
sanjaysubsAuthor
Gold ⭐️
January 3, 2023

Identical to this issue 

https://community.cloudera.com/t5/Support-Questions/NodeManagers-Unhealthy-mkdir-issue/m-p/54033/highlight/true

Translate
sanjaysubs
sanjaysubsAuthorAnswer
Gold ⭐️
January 3, 2023

OK I got it. In the yarn-site.xml I set the following in the Safety Valve

yarn.nodemanager.disk-health-checker.enable = false

Translate
AsimShaikh
Gold ⭐️
AsimShaikh
Gold ⭐️
January 3, 2023

yarn.nodemanager.disk-health-checker.enable is only to Enable or disable the disk health checker service and will not fix this issue.

Please share NM logs snippet for review.

 

Translate
sanjaysubs
sanjaysubsAuthor
Gold ⭐️
January 3, 2023

************************************************************/
2023-01-02 23:08:45,753 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NodeManager
STARTUP_MSG: host = hp8300one/10.0.0.3
STARTUP_MSG: args = []
STARTUP_MSG: version = 3.0.0-cdh6.3.4
STARTUP_MSG: classpath = /var/run/cloudera-scm-agent/process/311-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-io-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jul-to-slf4j-1.7.25.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/httpclient-4.5.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/accessors-smart-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/stax2-api-3.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-util-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/log4j-api-2.8.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/aws-java-sdk-bundle-1.11.563.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-configuration2-2.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-codec-1.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-mapper-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-lang3-3.7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jaxb-api-2.2.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/re2j-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-xdr-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-databind-2.9.10.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsch-0.1.54.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-servlet-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/httpcore-4.4.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/log4j-core-2.8.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-http-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-server-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-client-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-annotations-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-io-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jcip-annotations-1.0-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/xz-1.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-identity-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-security-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/woodstox-core-5.0.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-admin-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/javax.activation-api-1.2.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-compress-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/paranamer-2.8.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-core-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/slf4j-api-1.7.25.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-core-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/zookeeper.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-server-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-webapp-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-simplekdc-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/htrace-core4-4.1.0-incubating.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/nimbus-jose-jwt-7.9.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/netty-3.10.6.Final.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-common-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/snappy-java-1.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-core-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/audience-annotations-0.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-config-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/curator-framework-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-xml-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/asm-5.0.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-pkix-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/curator-client-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/json-smart-2.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/wildfly-openssl-1.0.4.Final.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-xc-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-beanutils-1.9.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-asn1-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-json-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/azure-data-lake-store-sdk-2.2.9.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsr311-api-1.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-servlet-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-server-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/avro.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/curator-recipes-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/metrics-core-3.0.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/javax.servlet-api-3.1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/logredactor-2.0.7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-crypto-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-nfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-kms.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-cascading3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-format.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-jackson.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-generator.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure-datalake-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-format-javadoc.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-format-sources.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-kms-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-thrift.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-pig-bundle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure-datalake.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-cascading.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-aws-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-nfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-annotations-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-protobuf.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-avro.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-encoding.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-aws.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-scala_2.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//gcs-connector-hadoop3-1.9.10-cdh6.3.4-shaded.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-pig.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-column.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-auth-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-hadoop.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//gcs-connector-hadoop3-shaded.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-auth.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-annotations.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/./:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-io-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/httpclient-4.5.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/okhttp-2.7.5.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/accessors-smart-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/stax2-api-3.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-util-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-jaxrs-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/json-simple-1.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-configuration2-2.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-codec-1.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-lang3-3.7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jaxb-api-2.2.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/re2j-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-xdr-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/avro-1.8.2-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-databind-2.9.10.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jsch-0.1.54.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-servlet-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/httpcore-4.4.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-http-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-server-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-client-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-annotations-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-io-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jcip-annotations-1.0-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/xz-1.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-identity-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-security-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/woodstox-core-5.0.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-admin-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/javax.activation-api-1.2.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-util-ajax-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-compress-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/paranamer-2.8.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-core-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-core-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-server-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-webapp-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-simplekdc-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/htrace-core4-4.1.0-incubating.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/nimbus-jose-jwt-7.9.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/netty-3.10.6.Final.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-common-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/snappy-java-1.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-core-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/audience-annotations-0.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-config-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/okio-1.6.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/curator-framework-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-xml-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/asm-5.0.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-pkix-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/curator-client-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/json-smart-2.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-xc-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-beanutils-1.9.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-asn1-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-json-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jsr311-api-1.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-servlet-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-server-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/curator-recipes-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/javax.servlet-api-3.1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-crypto-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-nfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-httpfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-nfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-httpfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-sls-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-extras.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-openstack.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-extras-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-distcp-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-gridmix-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-uploader-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archives-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//zstd-jni-1.3.8-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-sls.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-datajoin-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//ojalgo-43.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//kafka-clients-2.2.1-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//aliyun-sdk-oss-2.8.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-resourceestimator-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archive-logs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-rumen-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//lz4-java-1.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-aliyun-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-uploader.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archive-logs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-kafka-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-kafka.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-openstack-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//azure-keyvault-core-0.8.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//jdom-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-resourceestimator.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-streaming-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-aliyun.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archives.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//azure-storage-5.4.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/fst-2.50.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jackson-jaxrs-base-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/ehcache-3.3.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/HikariCP-java7-2.4.12.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/objenesis-1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/guice-servlet-4.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jersey-client-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jackson-jaxrs-json-provider-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/spark-2.4.0-cdh6.3.4-yarn-shuffle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/guice-4.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jackson-module-jaxb-annotations-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/bcpkix-jdk15on-1.60.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/bcprov-jdk15on-1.60.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/metrics-core-3.0.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jersey-guice-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-client-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-timeline-pluginstorage-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-api-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-client.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-registry-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-timeline-pluginstorage.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-registry.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-tests-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-router.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-router-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//timelineservice/hadoop-yarn-server-timelineservice-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//timelineservice/hadoop-yarn-server-timelineservice-hbase-3.0.0-cdh6.3.4.jar:/opt/cloudera/cm/lib/plugins/event-publish-7.8.1-shaded.jar:/opt/cloudera/cm/lib/plugins/tt-instrumentation-7.8.1.jar
STARTUP_MSG: build = http://github.com/cloudera/hadoop -r 7f337475e32e559b445985fb3f20a48f7e6927a2; compiled by 'jenkins' on 2020-10-28T06:38Z
STARTUP_MSG: java = 1.8.0_352
************************************************************/
2023-01-02 23:08:45,796 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: registered UNIX signal handlers for [TERM, HUP, INT]
2023-01-02 23:08:46,359 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: Using state database at /var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state for recovery
2023-01-02 23:08:46,390 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Recovering log #180
2023-01-02 23:08:46,483 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=2 #158

2023-01-02 23:08:46,484 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=3 #163

2023-01-02 23:08:46,484 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=0 #180

2023-01-02 23:08:46,491 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: Loaded NM state version info 1.2
2023-01-02 23:08:46,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.ResourcePluginManager: No Resource plugins found from configuration!
2023-01-02 23:08:46,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.ResourcePluginManager: Found Resource plugins from configuration: null
2023-01-02 23:08:46,634 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner.
2023-01-02 23:08:46,680 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher
2023-01-02 23:08:46,681 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher
2023-01-02 23:08:46,682 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$LocalizationEventHandlerWrapper
2023-01-02 23:08:46,683 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices
2023-01-02 23:08:46,683 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
2023-01-02 23:08:46,684 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher
2023-01-02 23:08:46,684 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerSchedulerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerScheduler
2023-01-02 23:08:46,700 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
2023-01-02 23:08:46,700 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.NodeManager
2023-01-02 23:08:46,736 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2023-01-02 23:08:46,818 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2023-01-02 23:08:46,818 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics system started
2023-01-02 23:08:46,844 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Disk Validator: yarn.nodemanager.disk-validator is loaded.
2023-01-02 23:08:46,855 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Disk Validator: yarn.nodemanager.disk-validator is loaded.
2023-01-02 23:08:46,860 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /media/sanjay/hdd02/yarn/nm error mkdir of /media/sanjay/hdd02 failed, removing from the list of valid directories.
2023-01-02 23:08:46,861 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /media/sanjay/hdd03/yarn/nm error mkdir of /media/sanjay/hdd03 failed, removing from the list of valid directories.
2023-01-02 23:08:46,890 INFO org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl: Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@146587a2
2023-01-02 23:08:46,893 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService
2023-01-02 23:08:46,895 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService
2023-01-02 23:08:46,895 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: AMRMProxyService is disabled
2023-01-02 23:08:46,895 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192
2023-01-02 23:08:46,910 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Disk Validator: yarn.nodemanager.disk-validator is loaded.
2023-01-02 23:08:46,916 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
2023-01-02 23:08:46,957 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Adding auxiliary service mapreduce_shuffle, "mapreduce_shuffle"
2023-01-02 23:08:47,014 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@4b45dcb8
2023-01-02 23:08:47,015 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Using ResourceCalculatorProcessTree : null
2023-01-02 23:08:47,016 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Physical memory check enabled: true
2023-01-02 23:08:47,016 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Virtual memory check enabled: false
2023-01-02 23:08:47,016 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: ContainersMonitor enabled: true
2023-01-02 23:08:47,018 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService: rollingMonitorInterval is set as -1. The log rolling monitoring interval is disabled. The logs will be aggregated after this application is finished.
2023-01-02 23:08:47,044 INFO org.apache.hadoop.conf.Configuration: resource-types.xml not found
2023-01-02 23:08:47,045 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'resource-types.xml'.
2023-01-02 23:08:47,046 INFO org.apache.hadoop.conf.Configuration: node-resources.xml not found
2023-01-02 23:08:47,047 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'node-resources.xml'.
2023-01-02 23:08:47,049 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Nodemanager resources is set to: <memory:20480, vCores:8>
2023-01-02 23:08:47,073 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Initialized nodemanager with : physical-memory=20480 virtual-memory=43008 virtual-cores=8
2023-01-02 23:08:47,151 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 2000 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2023-01-02 23:08:47,186 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8041
2023-01-02 23:08:47,225 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server
2023-01-02 23:08:47,229 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMContainerTokenSecretManager: Updating node address : hp8300one:8041
2023-01-02 23:08:47,236 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 500 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2023-01-02 23:08:47,242 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8040
2023-01-02 23:08:47,245 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server
2023-01-02 23:08:47,246 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2023-01-02 23:08:47,258 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8040: starting
2023-01-02 23:08:47,279 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer started on port 8040
2023-01-02 23:08:47,284 INFO org.apache.hadoop.mapred.ShuffleHandler: Using state database at /var/lib/hadoop-yarn/yarn-nm-recovery/nm-aux-services/mapreduce_shuffle/mapreduce_shuffle_state for recovery
2023-01-02 23:08:47,284 INFO org.apache.hadoop.mapred.ShuffleHandler$LevelDBLogger: Recovering log #22
2023-01-02 23:08:47,377 INFO org.apache.hadoop.mapred.ShuffleHandler$LevelDBLogger: Delete type=0 #22

2023-01-02 23:08:47,377 INFO org.apache.hadoop.mapred.ShuffleHandler$LevelDBLogger: Delete type=3 #21

2023-01-02 23:08:47,378 INFO org.apache.hadoop.mapred.ShuffleHandler: Loaded state DB schema version info 1.0
2023-01-02 23:08:47,394 INFO org.apache.hadoop.mapred.IndexCache: IndexCache created with max memory = 10485760
2023-01-02 23:08:47,430 INFO org.apache.hadoop.mapred.ShuffleHandler: mapreduce_shuffle listening on port 13562
2023-01-02 23:08:47,449 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerScheduler: Unknown event arrived at ContainerScheduler: EventType: RECOVERY_COMPLETED
2023-01-02 23:08:47,449 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2023-01-02 23:08:47,450 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8041: starting
2023-01-02 23:08:47,482 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: ContainerManager started at /10.0.0.3:8041
2023-01-02 23:08:47,482 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: ContainerManager bound to hp8300one/10.0.0.3:8041
2023-01-02 23:08:47,487 INFO org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer: Instantiating NMWebApp at hp8300one:8042
2023-01-02 23:08:47,515 INFO org.eclipse.jetty.util.log: Logging initialized @2499ms
2023-01-02 23:08:47,611 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2023-01-02 23:08:47,616 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.nodemanager is not defined
2023-01-02 23:08:47,625 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2023-01-02 23:08:47,627 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node
2023-01-02 23:08:47,627 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2023-01-02 23:08:47,627 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2023-01-02 23:08:47,629 INFO org.apache.hadoop.http.HttpServer2: Added filter authentication (class=org.apache.hadoop.security.authentication.server.AuthenticationFilter) to context node
2023-01-02 23:08:47,629 INFO org.apache.hadoop.http.HttpServer2: Added filter authentication (class=org.apache.hadoop.security.authentication.server.AuthenticationFilter) to context static
2023-01-02 23:08:47,629 INFO org.apache.hadoop.http.HttpServer2: Added filter authentication (class=org.apache.hadoop.security.authentication.server.AuthenticationFilter) to context logs
2023-01-02 23:08:47,631 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /node/*
2023-01-02 23:08:47,632 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2023-01-02 23:08:48,178 INFO org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2023-01-02 23:08:48,182 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8042
2023-01-02 23:08:48,183 INFO org.eclipse.jetty.server.Server: jetty-9.3.25.v20180904, build timestamp: 2018-09-04T14:11:46-07:00, git hash: 3ce520221d0240229c862b122d2b06c12a625732
2023-01-02 23:08:48,306 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2023-01-02 23:08:48,310 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@7c22d4f{/logs,file:///var/log/hadoop-yarn/,AVAILABLE}
2023-01-02 23:08:48,324 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@51e4ccb3{/static,jar:file:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/jars/hadoop-yarn-common-3.0.0-cdh6.3.4.jar!/webapps/static,AVAILABLE}
2023-01-02 23:08:49,494 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@6cd164a6{/,file:///tmp/jetty-hp8300one-8042-node-_-any-9073807784737228590.dir/webapp/,AVAILABLE}{/node}
2023-01-02 23:08:49,499 INFO org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@474c9131{HTTP/1.1,[http/1.1]}{hp8300one:8042}
2023-01-02 23:08:49,499 INFO org.eclipse.jetty.server.Server: Started @4482ms
2023-01-02 23:08:49,499 INFO org.apache.hadoop.yarn.webapp.WebApps: Web app node started at 8042
2023-01-02 23:08:49,500 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node ID assigned is : hp8300one:8041
2023-01-02 23:08:49,501 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor
2023-01-02 23:08:49,507 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at hp8300one/10.0.0.3:8031
2023-01-02 23:08:49,548 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 0 NM container statuses: []
2023-01-02 23:08:49,555 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Registering with RM using containers :[]
2023-01-02 23:08:49,612 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMContainerTokenSecretManager: Rolling master-key for container-tokens, got key with id 1232403110
2023-01-02 23:08:49,613 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMTokenSecretManagerInNM: Rolling master-key for container-tokens, got key with id 436856332
2023-01-02 23:08:49,614 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Registered with ResourceManager as hp8300one:8041 with total resource of <memory:20480, vCores:8>
2023-01-02 23:09:56,227 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1672729727095_0001_01_000001 by user sanjay
2023-01-02 23:09:56,271 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Creating a new application reference for app application_1672729727095_0001
2023-01-02 23:09:56,278 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from NEW to INITING
2023-01-02 23:09:56,285 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay IP=10.0.0.3 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1672729727095_0001 CONTAINERID=container_1672729727095_0001_01_000001
2023-01-02 23:09:56,673 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Adding container_1672729727095_0001_01_000001 to application application_1672729727095_0001
2023-01-02 23:09:56,679 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from INITING to RUNNING
2023-01-02 23:09:56,686 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0001_01_000001 transitioned from NEW to LOCALIZING
2023-01-02 23:09:56,686 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_INIT for appId application_1672729727095_0001
2023-01-02 23:09:56,694 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Downloading public resource: { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null }
2023-01-02 23:09:56,698 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Local path for public localization is not found. May be disks failed.
org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories.
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:589)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$PublicLocalizer.addResource(ResourceLocalizationService.java:883)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:781)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:723)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:750)
2023-01-02 23:09:56,699 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_1672729727095_0001_01_000001
2023-01-02 23:09:56,701 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer failed for container_1672729727095_0001_01_000001
org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories.
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:133)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:117)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:584)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1205)
2023-01-02 23:09:56,701 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0001_01_000001 transitioned from LOCALIZING to LOCALIZATION_FAILED
2023-01-02 23:09:56,702 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: Container container_1672729727095_0001_01_000001 sent RELEASE event on a resource request { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } not present in cache.
2023-01-02 23:09:56,702 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: LOCALIZATION_FAILED APPID=application_1672729727095_0001 CONTAINERID=container_1672729727095_0001_01_000001
2023-01-02 23:09:56,711 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0001_01_000001 transitioned from LOCALIZATION_FAILED to DONE
2023-01-02 23:09:56,711 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1672729727095_0001_01_000001 from application application_1672729727095_0001
2023-01-02 23:09:56,713 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1672729727095_0001_01_000001
2023-01-02 23:09:56,713 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1672729727095_0001_01_000001 for log-aggregation
2023-01-02 23:09:56,713 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1672729727095_0001
2023-01-02 23:09:56,717 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Execution exception when running task in DeletionService #0
2023-01-02 23:09:56,717 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Caught exception in thread DeletionService #0:
java.lang.NullPointerException: path cannot be null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:270)
at org.apache.hadoop.fs.FileContext.delete(FileContext.java:768)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.deletion.task.FileDeletionTask.run(FileDeletionTask.java:109)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-01-02 23:09:57,718 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1672729727095_0001_01_000001]
2023-01-02 23:09:58,722 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP
2023-01-02 23:09:58,723 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1672729727095_0001
2023-01-02 23:09:58,726 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
2023-01-02 23:09:58,726 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Application just finished : application_1672729727095_0001
2023-01-02 23:09:58,836 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_1672729727095_0001_01_000001. Current good log dirs are /yarn/container-logs
2023-01-02 23:18:47,233 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0
2023-01-02 23:28:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0
2023-01-02 23:29:42,418 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1672729727095_0002_01_000001 by user sanjay
2023-01-02 23:29:42,420 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Creating a new application reference for app application_1672729727095_0002
2023-01-02 23:29:42,420 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from NEW to INITING
2023-01-02 23:29:42,420 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay IP=10.0.0.3 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1672729727095_0002 CONTAINERID=container_1672729727095_0002_01_000001
2023-01-02 23:29:42,459 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Adding container_1672729727095_0002_01_000001 to application application_1672729727095_0002
2023-01-02 23:29:42,459 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from INITING to RUNNING
2023-01-02 23:29:42,461 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from NEW to LOCALIZING
2023-01-02 23:29:42,461 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_INIT for appId application_1672729727095_0002
2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Downloading public resource: { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null }
2023-01-02 23:29:42,462 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Local path for public localization is not found. May be disks failed.
org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories.
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:589)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$PublicLocalizer.addResource(ResourceLocalizationService.java:883)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:781)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:723)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:750)
2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_1672729727095_0002_01_000001
2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer failed for container_1672729727095_0002_01_000001
org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories.
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:133)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:117)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:584)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1205)
2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from LOCALIZING to LOCALIZATION_FAILED
2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: Container container_1672729727095_0002_01_000001 sent RELEASE event on a resource request { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } not present in cache.
2023-01-02 23:29:42,463 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Execution exception when running task in DeletionService #0
2023-01-02 23:29:42,464 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Caught exception in thread DeletionService #0:
java.lang.NullPointerException: path cannot be null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:270)
at org.apache.hadoop.fs.FileContext.delete(FileContext.java:768)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.deletion.task.FileDeletionTask.run(FileDeletionTask.java:109)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-01-02 23:29:42,464 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: LOCALIZATION_FAILED APPID=application_1672729727095_0002 CONTAINERID=container_1672729727095_0002_01_000001
2023-01-02 23:29:42,465 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from LOCALIZATION_FAILED to DONE
2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1672729727095_0002_01_000001 from application application_1672729727095_0002
2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1672729727095_0002_01_000001
2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1672729727095_0002_01_000001 for log-aggregation
2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1672729727095_0002
2023-01-02 23:29:43,470 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1672729727095_0002_01_000001]
2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP
2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1672729727095_0002
2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Application just finished : application_1672729727095_0002
2023-01-02 23:29:43,485 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_1672729727095_0002_01_000001. Current good log dirs are /yarn/container-logs
2023-01-02 23:38:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0
2023-01-02 23:48:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0
2023-01-02 23:58:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0

Translate
sanjaysubs
sanjaysubsAuthor
Gold ⭐️
January 3, 2023

Screen Shot 2023-01-03 at 9.34.09 AM.png

Translate
sanjaysubs
sanjaysubsAuthor
Gold ⭐️
January 3, 2023

 I am getting errors while running a simple MR job 

 

2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Downloading public resource: { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null }
2023-01-02 23:29:42,462 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Local path for public localization is not found. May be disks failed.
org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories.
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:589)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$PublicLocalizer.addResource(ResourceLocalizationService.java:883)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:781)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:723)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:750)
2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_1672729727095_0002_01_000001
2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer failed for container_1672729727095_0002_01_000001
org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories.
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:133)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:117)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:584)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1205)
2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from LOCALIZING to LOCALIZATION_FAILED
2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: Container container_1672729727095_0002_01_000001 sent RELEASE event on a resource request { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } not present in cache.
2023-01-02 23:29:42,463 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Execution exception when running task in DeletionService #0
2023-01-02 23:29:42,464 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Caught exception in thread DeletionService #0:
java.lang.NullPointerException: path cannot be null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:270)
at org.apache.hadoop.fs.FileContext.delete(FileContext.java:768)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.deletion.task.FileDeletionTask.run(FileDeletionTask.java:109)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-01-02 23:29:42,464 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: LOCALIZATION_FAILED APPID=application_1672729727095_0002 CONTAINERID=container_1672729727095_0002_01_000001

Translate
Terms & ConditionsAccessibility statement