@Kartik_Agarwal  Unfortunately this does not solve the issue. I just request a very clear step by step explanation of where I need to specify all the variables needed to run a MR or a Hive job. I have been using CDH since 2011 but this is the first time I cannot even run a Wordcount program successfully !!! Very disappointed and discouraged.     When I go to the Configuration file for the job that failed from the WebUI it shows me this for the property <property> <name> yarn.nodemanager.local-dirs </name> <value> ${hadoop.tmp.dir}/nm-local-dir </value> <final> false </final> <source> yarn-default.xml </source> </property>   However in Cloudera Manager the values are differently shown     Also the local directories specified by "yarn.nodemanager.local-dirs" are empty ls -latr /media/sanjay/hdd0[23]/yarn/nm /media/sanjay/hdd03/yarn/nm: total 8 drwxrwxrwx 3 yarn hadoop 4096 Jan 2 11:25 .. drwxr-xr-x 2 yarn hadoop 4096 Jan 2 11:25 . /media/sanjay/hdd02/yarn/nm: total 8 drwxrwxrwx 3 yarn hadoop 4096 Jan 2 11:25 .. drwxr-xr-x 2 yarn hadoop 4096 Jan 2 11:25 .   My "hadoop.tmp.dir" is defined here ... View more

I am getting a similar error.  https://community.cloudera.com/t5/Support-Questions/Map-Reduce-Job-fails-exited-with-exitCode-1000/m-p/361335#M238562 ... View more

This issue stated here is pretty much same as what I am facing https://stackoverflow.com/questions/71684661/no-disk-space-allocated-to-hdfs-filesystem ... View more

 I am getting errors while running a simple MR job    2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Downloading public resource: { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } 2023-01-02 23:29:42,462 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Local path for public localization is not found. May be disks failed. org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:589) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$PublicLocalizer.addResource(ResourceLocalizationService.java:883) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:781) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:723) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126) at java.lang.Thread.run(Thread.java:750) 2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_1672729727095_0002_01_000001 2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer failed for container_1672729727095_0002_01_000001 org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:133) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:117) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:584) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1205) 2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from LOCALIZING to LOCALIZATION_FAILED 2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: Container container_1672729727095_0002_01_000001 sent RELEASE event on a resource request { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } not present in cache. 2023-01-02 23:29:42,463 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Execution exception when running task in DeletionService #0 2023-01-02 23:29:42,464 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Caught exception in thread DeletionService #0: java.lang.NullPointerException: path cannot be null at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204) at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:270) at org.apache.hadoop.fs.FileContext.delete(FileContext.java:768) at org.apache.hadoop.yarn.server.nodemanager.containermanager.deletion.task.FileDeletionTask.run(FileDeletionTask.java:109) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-01-02 23:29:42,464 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: LOCALIZATION_FAILED APPID=application_1672729727095_0002 CONTAINERID=container_1672729727095_0002_01_000001 ... View more

************************************************************/ 2023-01-02 23:08:45,753 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NodeManager STARTUP_MSG: host = hp8300one/10.0.0.3 STARTUP_MSG: args = [] STARTUP_MSG: version = 3.0.0-cdh6.3.4 STARTUP_MSG: classpath = /var/run/cloudera-scm-agent/process/311-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-io-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jul-to-slf4j-1.7.25.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/httpclient-4.5.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/accessors-smart-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/stax2-api-3.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-util-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/log4j-api-2.8.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/aws-java-sdk-bundle-1.11.563.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-configuration2-2.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-codec-1.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-mapper-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-lang3-3.7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jaxb-api-2.2.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/re2j-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-xdr-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-databind-2.9.10.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsch-0.1.54.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-servlet-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/httpcore-4.4.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/log4j-core-2.8.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-http-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-server-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-client-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-annotations-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-io-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jcip-annotations-1.0-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/xz-1.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-identity-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-security-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/woodstox-core-5.0.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-admin-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/javax.activation-api-1.2.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-compress-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/paranamer-2.8.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-core-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/slf4j-api-1.7.25.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-core-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/zookeeper.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-server-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-webapp-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-simplekdc-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/htrace-core4-4.1.0-incubating.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/nimbus-jose-jwt-7.9.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/netty-3.10.6.Final.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-common-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/snappy-java-1.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-core-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/audience-annotations-0.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-config-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/curator-framework-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-xml-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/asm-5.0.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-pkix-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/curator-client-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/json-smart-2.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/wildfly-openssl-1.0.4.Final.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jackson-xc-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/commons-beanutils-1.9.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerby-asn1-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jersey-json-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/azure-data-lake-store-sdk-2.2.9.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jsr311-api-1.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-servlet-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/jetty-server-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/avro.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/curator-recipes-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/metrics-core-3.0.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/javax.servlet-api-3.1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/logredactor-2.0.7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/lib/kerb-crypto-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-nfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-kms.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-cascading3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-format.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-jackson.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-generator.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure-datalake-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-format-javadoc.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-format-sources.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-kms-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-thrift.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-pig-bundle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure-datalake.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-cascading.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-aws-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-nfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-annotations-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-protobuf.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-avro.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-encoding.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-aws.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-scala_2.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//gcs-connector-hadoop3-1.9.10-cdh6.3.4-shaded.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-pig.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-azure-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-column.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-auth-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//parquet-hadoop.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//gcs-connector-hadoop3-shaded.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-auth.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop/.//hadoop-annotations.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/./:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-io-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/httpclient-4.5.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/okhttp-2.7.5.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/accessors-smart-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/stax2-api-3.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-util-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-jaxrs-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/json-simple-1.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-configuration2-2.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-codec-1.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-lang3-3.7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jaxb-api-2.2.11.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/re2j-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-xdr-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/avro-1.8.2-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-databind-2.9.10.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jsch-0.1.54.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-servlet-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/httpcore-4.4.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-http-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-server-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-client-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-annotations-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-io-2.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jcip-annotations-1.0-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/xz-1.6.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-identity-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-security-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/woodstox-core-5.0.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-admin-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/javax.activation-api-1.2.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-util-ajax-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-compress-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/paranamer-2.8.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-core-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-core-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-server-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-webapp-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-simplekdc-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/htrace-core4-4.1.0-incubating.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/nimbus-jose-jwt-7.9.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/netty-3.10.6.Final.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-common-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/snappy-java-1.1.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-core-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/audience-annotations-0.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-config-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/okio-1.6.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/curator-framework-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-xml-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/asm-5.0.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-pkix-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/curator-client-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/json-smart-2.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jackson-xc-1.9.13.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/commons-beanutils-1.9.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerby-asn1-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jersey-json-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jsr311-api-1.1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-util-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-servlet-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/jetty-server-9.3.25.v20180904.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/curator-recipes-2.12.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/javax.servlet-api-3.1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/lib/kerb-crypto-1.0.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-nfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-httpfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-nfs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-client.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-httpfs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-sls-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-extras.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-openstack.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-extras-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-distcp-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-gridmix-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-uploader-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archives-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//zstd-jni-1.3.8-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-sls.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-datajoin-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//ojalgo-43.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//kafka-clients-2.2.1-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//aliyun-sdk-oss-2.8.3.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-resourceestimator-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archive-logs-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-rumen-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//lz4-java-1.5.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-aliyun-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-uploader.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archive-logs.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-kafka-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-kafka.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-openstack-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//azure-keyvault-core-0.8.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//jdom-1.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.4-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-resourceestimator.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-streaming-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-aliyun.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-archives.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-mapreduce/.//azure-storage-5.4.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/fst-2.50.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jackson-jaxrs-base-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/ehcache-3.3.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/HikariCP-java7-2.4.12.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/objenesis-1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/guice-servlet-4.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jersey-client-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jackson-jaxrs-json-provider-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/spark-2.4.0-cdh6.3.4-yarn-shuffle.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/guice-4.0.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jackson-module-jaxb-annotations-2.9.10.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/bcpkix-jdk15on-1.60.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/bcprov-jdk15on-1.60.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/metrics-core-3.0.1.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/lib/jersey-guice-1.19.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-client-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-timeline-pluginstorage-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-api-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-client.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-registry-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-timeline-pluginstorage.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-registry.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-tests-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-router.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-common-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-router-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//timelineservice/hadoop-yarn-server-timelineservice-3.0.0-cdh6.3.4.jar:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/lib/hadoop-yarn/.//timelineservice/hadoop-yarn-server-timelineservice-hbase-3.0.0-cdh6.3.4.jar:/opt/cloudera/cm/lib/plugins/event-publish-7.8.1-shaded.jar:/opt/cloudera/cm/lib/plugins/tt-instrumentation-7.8.1.jar STARTUP_MSG: build = http://github.com/cloudera/hadoop -r 7f337475e32e559b445985fb3f20a48f7e6927a2; compiled by 'jenkins' on 2020-10-28T06:38Z STARTUP_MSG: java = 1.8.0_352 ************************************************************/ 2023-01-02 23:08:45,796 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: registered UNIX signal handlers for [TERM, HUP, INT] 2023-01-02 23:08:46,359 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: Using state database at /var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state for recovery 2023-01-02 23:08:46,390 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Recovering log #180 2023-01-02 23:08:46,483 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=2 #158 2023-01-02 23:08:46,484 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=3 #163 2023-01-02 23:08:46,484 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=0 #180 2023-01-02 23:08:46,491 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: Loaded NM state version info 1.2 2023-01-02 23:08:46,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.ResourcePluginManager: No Resource plugins found from configuration! 2023-01-02 23:08:46,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.ResourcePluginManager: Found Resource plugins from configuration: null 2023-01-02 23:08:46,634 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. 2023-01-02 23:08:46,680 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2023-01-02 23:08:46,681 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2023-01-02 23:08:46,682 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$LocalizationEventHandlerWrapper 2023-01-02 23:08:46,683 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2023-01-02 23:08:46,683 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2023-01-02 23:08:46,684 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2023-01-02 23:08:46,684 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerSchedulerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerScheduler 2023-01-02 23:08:46,700 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2023-01-02 23:08:46,700 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.NodeManager 2023-01-02 23:08:46,736 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties 2023-01-02 23:08:46,818 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2023-01-02 23:08:46,818 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics system started 2023-01-02 23:08:46,844 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Disk Validator: yarn.nodemanager.disk-validator is loaded. 2023-01-02 23:08:46,855 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Disk Validator: yarn.nodemanager.disk-validator is loaded. 2023-01-02 23:08:46,860 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /media/sanjay/hdd02/yarn/nm error mkdir of /media/sanjay/hdd02 failed, removing from the list of valid directories. 2023-01-02 23:08:46,861 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /media/sanjay/hdd03/yarn/nm error mkdir of /media/sanjay/hdd03 failed, removing from the list of valid directories. 2023-01-02 23:08:46,890 INFO org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl: Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@146587a2 2023-01-02 23:08:46,893 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService 2023-01-02 23:08:46,895 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService 2023-01-02 23:08:46,895 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: AMRMProxyService is disabled 2023-01-02 23:08:46,895 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192 2023-01-02 23:08:46,910 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Disk Validator: yarn.nodemanager.disk-validator is loaded. 2023-01-02 23:08:46,916 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2023-01-02 23:08:46,957 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Adding auxiliary service mapreduce_shuffle, "mapreduce_shuffle" 2023-01-02 23:08:47,014 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.ResourceCalculatorPlugin@4b45dcb8 2023-01-02 23:08:47,015 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Using ResourceCalculatorProcessTree : null 2023-01-02 23:08:47,016 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Physical memory check enabled: true 2023-01-02 23:08:47,016 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Virtual memory check enabled: false 2023-01-02 23:08:47,016 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: ContainersMonitor enabled: true 2023-01-02 23:08:47,018 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService: rollingMonitorInterval is set as -1. The log rolling monitoring interval is disabled. The logs will be aggregated after this application is finished. 2023-01-02 23:08:47,044 INFO org.apache.hadoop.conf.Configuration: resource-types.xml not found 2023-01-02 23:08:47,045 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'resource-types.xml'. 2023-01-02 23:08:47,046 INFO org.apache.hadoop.conf.Configuration: node-resources.xml not found 2023-01-02 23:08:47,047 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'node-resources.xml'. 2023-01-02 23:08:47,049 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Nodemanager resources is set to: <memory:20480, vCores:8> 2023-01-02 23:08:47,073 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Initialized nodemanager with : physical-memory=20480 virtual-memory=43008 virtual-cores=8 2023-01-02 23:08:47,151 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 2000 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler 2023-01-02 23:08:47,186 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8041 2023-01-02 23:08:47,225 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2023-01-02 23:08:47,229 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMContainerTokenSecretManager: Updating node address : hp8300one:8041 2023-01-02 23:08:47,236 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 500 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler 2023-01-02 23:08:47,242 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8040 2023-01-02 23:08:47,245 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2023-01-02 23:08:47,246 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2023-01-02 23:08:47,258 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8040: starting 2023-01-02 23:08:47,279 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer started on port 8040 2023-01-02 23:08:47,284 INFO org.apache.hadoop.mapred.ShuffleHandler: Using state database at /var/lib/hadoop-yarn/yarn-nm-recovery/nm-aux-services/mapreduce_shuffle/mapreduce_shuffle_state for recovery 2023-01-02 23:08:47,284 INFO org.apache.hadoop.mapred.ShuffleHandler$LevelDBLogger: Recovering log #22 2023-01-02 23:08:47,377 INFO org.apache.hadoop.mapred.ShuffleHandler$LevelDBLogger: Delete type=0 #22 2023-01-02 23:08:47,377 INFO org.apache.hadoop.mapred.ShuffleHandler$LevelDBLogger: Delete type=3 #21 2023-01-02 23:08:47,378 INFO org.apache.hadoop.mapred.ShuffleHandler: Loaded state DB schema version info 1.0 2023-01-02 23:08:47,394 INFO org.apache.hadoop.mapred.IndexCache: IndexCache created with max memory = 10485760 2023-01-02 23:08:47,430 INFO org.apache.hadoop.mapred.ShuffleHandler: mapreduce_shuffle listening on port 13562 2023-01-02 23:08:47,449 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerScheduler: Unknown event arrived at ContainerScheduler: EventType: RECOVERY_COMPLETED 2023-01-02 23:08:47,449 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2023-01-02 23:08:47,450 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8041: starting 2023-01-02 23:08:47,482 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: ContainerManager started at /10.0.0.3:8041 2023-01-02 23:08:47,482 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: ContainerManager bound to hp8300one/10.0.0.3:8041 2023-01-02 23:08:47,487 INFO org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer: Instantiating NMWebApp at hp8300one:8042 2023-01-02 23:08:47,515 INFO org.eclipse.jetty.util.log: Logging initialized @2499ms 2023-01-02 23:08:47,611 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2023-01-02 23:08:47,616 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.nodemanager is not defined 2023-01-02 23:08:47,625 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2023-01-02 23:08:47,627 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2023-01-02 23:08:47,627 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-01-02 23:08:47,627 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-01-02 23:08:47,629 INFO org.apache.hadoop.http.HttpServer2: Added filter authentication (class=org.apache.hadoop.security.authentication.server.AuthenticationFilter) to context node 2023-01-02 23:08:47,629 INFO org.apache.hadoop.http.HttpServer2: Added filter authentication (class=org.apache.hadoop.security.authentication.server.AuthenticationFilter) to context static 2023-01-02 23:08:47,629 INFO org.apache.hadoop.http.HttpServer2: Added filter authentication (class=org.apache.hadoop.security.authentication.server.AuthenticationFilter) to context logs 2023-01-02 23:08:47,631 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /node/* 2023-01-02 23:08:47,632 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/* 2023-01-02 23:08:48,178 INFO org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules 2023-01-02 23:08:48,182 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8042 2023-01-02 23:08:48,183 INFO org.eclipse.jetty.server.Server: jetty-9.3.25.v20180904, build timestamp: 2018-09-04T14:11:46-07:00, git hash: 3ce520221d0240229c862b122d2b06c12a625732 2023-01-02 23:08:48,306 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2023-01-02 23:08:48,310 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@7c22d4f{/logs,file:///var/log/hadoop-yarn/,AVAILABLE} 2023-01-02 23:08:48,324 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@51e4ccb3{/static,jar:file:/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/jars/hadoop-yarn-common-3.0.0-cdh6.3.4.jar!/webapps/static,AVAILABLE} 2023-01-02 23:08:49,494 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@6cd164a6{/,file:///tmp/jetty-hp8300one-8042-node-_-any-9073807784737228590.dir/webapp/,AVAILABLE}{/node} 2023-01-02 23:08:49,499 INFO org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@474c9131{HTTP/1.1,[http/1.1]}{hp8300one:8042} 2023-01-02 23:08:49,499 INFO org.eclipse.jetty.server.Server: Started @4482ms 2023-01-02 23:08:49,499 INFO org.apache.hadoop.yarn.webapp.WebApps: Web app node started at 8042 2023-01-02 23:08:49,500 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node ID assigned is : hp8300one:8041 2023-01-02 23:08:49,501 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor 2023-01-02 23:08:49,507 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at hp8300one/10.0.0.3:8031 2023-01-02 23:08:49,548 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 0 NM container statuses: [] 2023-01-02 23:08:49,555 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Registering with RM using containers :[] 2023-01-02 23:08:49,612 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMContainerTokenSecretManager: Rolling master-key for container-tokens, got key with id 1232403110 2023-01-02 23:08:49,613 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMTokenSecretManagerInNM: Rolling master-key for container-tokens, got key with id 436856332 2023-01-02 23:08:49,614 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Registered with ResourceManager as hp8300one:8041 with total resource of <memory:20480, vCores:8> 2023-01-02 23:09:56,227 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1672729727095_0001_01_000001 by user sanjay 2023-01-02 23:09:56,271 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Creating a new application reference for app application_1672729727095_0001 2023-01-02 23:09:56,278 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from NEW to INITING 2023-01-02 23:09:56,285 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay IP=10.0.0.3 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1672729727095_0001 CONTAINERID=container_1672729727095_0001_01_000001 2023-01-02 23:09:56,673 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Adding container_1672729727095_0001_01_000001 to application application_1672729727095_0001 2023-01-02 23:09:56,679 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from INITING to RUNNING 2023-01-02 23:09:56,686 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0001_01_000001 transitioned from NEW to LOCALIZING 2023-01-02 23:09:56,686 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_INIT for appId application_1672729727095_0001 2023-01-02 23:09:56,694 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Downloading public resource: { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } 2023-01-02 23:09:56,698 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Local path for public localization is not found. May be disks failed. org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:589) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$PublicLocalizer.addResource(ResourceLocalizationService.java:883) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:781) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:723) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126) at java.lang.Thread.run(Thread.java:750) 2023-01-02 23:09:56,699 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_1672729727095_0001_01_000001 2023-01-02 23:09:56,701 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer failed for container_1672729727095_0001_01_000001 org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:133) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:117) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:584) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1205) 2023-01-02 23:09:56,701 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0001_01_000001 transitioned from LOCALIZING to LOCALIZATION_FAILED 2023-01-02 23:09:56,702 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: Container container_1672729727095_0001_01_000001 sent RELEASE event on a resource request { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } not present in cache. 2023-01-02 23:09:56,702 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: LOCALIZATION_FAILED APPID=application_1672729727095_0001 CONTAINERID=container_1672729727095_0001_01_000001 2023-01-02 23:09:56,711 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0001_01_000001 transitioned from LOCALIZATION_FAILED to DONE 2023-01-02 23:09:56,711 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1672729727095_0001_01_000001 from application application_1672729727095_0001 2023-01-02 23:09:56,713 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1672729727095_0001_01_000001 2023-01-02 23:09:56,713 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1672729727095_0001_01_000001 for log-aggregation 2023-01-02 23:09:56,713 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1672729727095_0001 2023-01-02 23:09:56,717 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Execution exception when running task in DeletionService #0 2023-01-02 23:09:56,717 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Caught exception in thread DeletionService #0: java.lang.NullPointerException: path cannot be null at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204) at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:270) at org.apache.hadoop.fs.FileContext.delete(FileContext.java:768) at org.apache.hadoop.yarn.server.nodemanager.containermanager.deletion.task.FileDeletionTask.run(FileDeletionTask.java:109) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-01-02 23:09:57,718 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1672729727095_0001_01_000001] 2023-01-02 23:09:58,722 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2023-01-02 23:09:58,723 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1672729727095_0001 2023-01-02 23:09:58,726 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2023-01-02 23:09:58,726 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Application just finished : application_1672729727095_0001 2023-01-02 23:09:58,836 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_1672729727095_0001_01_000001. Current good log dirs are /yarn/container-logs 2023-01-02 23:18:47,233 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0 2023-01-02 23:28:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0 2023-01-02 23:29:42,418 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1672729727095_0002_01_000001 by user sanjay 2023-01-02 23:29:42,420 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Creating a new application reference for app application_1672729727095_0002 2023-01-02 23:29:42,420 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from NEW to INITING 2023-01-02 23:29:42,420 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay IP=10.0.0.3 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1672729727095_0002 CONTAINERID=container_1672729727095_0002_01_000001 2023-01-02 23:29:42,459 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Adding container_1672729727095_0002_01_000001 to application application_1672729727095_0002 2023-01-02 23:29:42,459 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from INITING to RUNNING 2023-01-02 23:29:42,461 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from NEW to LOCALIZING 2023-01-02 23:29:42,461 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_INIT for appId application_1672729727095_0002 2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Downloading public resource: { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } 2023-01-02 23:29:42,462 ERROR org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Local path for public localization is not found. May be disks failed. org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:589) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$PublicLocalizer.addResource(ResourceLocalizationService.java:883) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:781) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:723) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126) at java.lang.Thread.run(Thread.java:750) 2023-01-02 23:29:42,462 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_1672729727095_0002_01_000001 2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer failed for container_1672729727095_0002_01_000001 org.apache.hadoop.util.DiskChecker$DiskErrorException: No space available in any of the local directories. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:400) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:133) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:117) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:584) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1205) 2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from LOCALIZING to LOCALIZATION_FAILED 2023-01-02 23:29:42,463 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: Container container_1672729727095_0002_01_000001 sent RELEASE event on a resource request { hdfs://hp8300one:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.4-mr-framework.tar.gz, 1672446065301, ARCHIVE, null } not present in cache. 2023-01-02 23:29:42,463 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Execution exception when running task in DeletionService #0 2023-01-02 23:29:42,464 WARN org.apache.hadoop.util.concurrent.ExecutorHelper: Caught exception in thread DeletionService #0: java.lang.NullPointerException: path cannot be null at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204) at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:270) at org.apache.hadoop.fs.FileContext.delete(FileContext.java:768) at org.apache.hadoop.yarn.server.nodemanager.containermanager.deletion.task.FileDeletionTask.run(FileDeletionTask.java:109) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-01-02 23:29:42,464 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=sanjay OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: LOCALIZATION_FAILED APPID=application_1672729727095_0002 CONTAINERID=container_1672729727095_0002_01_000001 2023-01-02 23:29:42,465 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1672729727095_0002_01_000001 transitioned from LOCALIZATION_FAILED to DONE 2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1672729727095_0002_01_000001 from application application_1672729727095_0002 2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1672729727095_0002_01_000001 2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1672729727095_0002_01_000001 for log-aggregation 2023-01-02 23:29:42,466 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1672729727095_0002 2023-01-02 23:29:43,470 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1672729727095_0002_01_000001] 2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1672729727095_0002 2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1672729727095_0002 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2023-01-02 23:29:43,471 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Application just finished : application_1672729727095_0002 2023-01-02 23:29:43,485 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_1672729727095_0002_01_000001. Current good log dirs are /yarn/container-logs 2023-01-02 23:38:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0 2023-01-02 23:48:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0 2023-01-02 23:58:47,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0 ... View more

Identical to this issue  https://community.cloudera.com/t5/Support-Questions/NodeManagers-Unhealthy-mkdir-issue/m-p/54033/highlight/true ... View more

2023-01-01 17:31:58,726 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 2/2 local-dirs have errors: [ /media/sanjay/hdd02/yarn/nm : Cannot create directory : /media/sanjay/hdd02/yarn/nm, error mkdir of /media/sanjay/hdd02 failed , /media/sanjay/hdd03/yarn/nm : Cannot create directory : /media/sanjay/hdd03/yarn/nm, error mkdir of /media/sanjay/hdd03 failed ] ... View more

U r right. The errors are as follows    [ /media/sanjay/hdd02/yarn/nm : Cannot create directory: /media/sanjay/hdd02/yarn/nm , /media/sanjay/hdd03/yarn/nm : Cannot create directory: /media/sanjay/hdd03/yarn/nm ]   I have tried changing permissions / owners of the folders (yarn:hadoop) and in ultimate desperation made the folders 777 as well just to see if the problem gets solved. No luck yet...Not sure how to solve this. Running out of ideas fast LOL ... View more

You know this 🙂 . I unfortunately don't. Since I did not find any documentation on how to actually find and use the allkeys.asc file. So I decided to be innovative and installed the agents myself. ... View more

I find only one ERROR in the /var/log/cloudera-scm-server/cloudera-scm-server.log   sudo grep ERROR /var/log/cloudera-scm-server/cloudera-scm-server.log | cut -d ' ' -f 3- | sort -u ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Failed to download manifest. Status code: 404 URI: https://archive.cloudera.com/accumulo6/6.1.0/parcels/manifest.json/   ... View more

@parasHi To clarify out of the three nodes installation succeeded on one node (the node that has CM manager daemon running). The installation failed on the other two nodes. I will and find out details about allkeys.asc and see where to exactly put that file on each node...   I am just puzzled that the install worked on one node but is failing on the remaining 2.    I will also re-check the CM server logs to see if there are any clues during the startup that might help me solve this problem.   I have been using Cloudera CM and CDH since 2011 ! That time I had a handle in the community called sanjumani. Now after this Cloudera rehaul past two years I cannot get to that account anymore. So sanjaysubs is my new handle   Thanks   ... View more

To confirm this is a 3 node hadoop cluster I am trying to setup with Cloudera. - Ubuntu 18.04.5 LTS - I installed Anaconda3-2022.10-Linux-x86_64.sh on each node - On each node I installed openjdk version "1.8.0_352"   ... View more