Hello Everyone,   I'm trying to distcp from an HDP 3.1.5 cluster (non-kerberized) to CDP 7.1.5 cluster (kerberized).   I'm running the distcp command on the secure cluster like follows: hadoop distcp -Ddfs.client.use.datanode.hostname=true -Ddfs.datanode.use.datanode.hostname=true -Dipc.client.fallback-to-simple-auth-allowed=true webhdfs://<source-nameservice>/path/to/source/folder webhdfs://<destination-nameservice>/path/to/destination/folder   It starts fine and then fails after 40 mins (Map progress reaches 97% and then gets stuck).   When I looked into the task attempt logs, I find this error:     2021-04-19 17:00:30,916 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration 2021-04-19 17:00:30,985 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties 2021-04-19 17:00:31,038 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2021-04-19 17:00:31,038 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started 2021-04-19 17:00:31,104 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: [Kind: mapreduce.job, Service: job_1618819500492_0009, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@233795b6), Kind: WEBHDFS delegation, Service: ha-webhdfs:<destination-nameservice>, Ident: (token for megh.vidani: WEBHDFS delegation owner=megh.vidani, renewer=yarn, realUser=, issueDate=1618831821242, maxDate=1619436621242, sequenceNumber=322, masterKeyId=30), Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:<destination-nameservice>, Ident: (token for megh.vidani: HDFS_DELEGATION_TOKEN owner=megh.vidani@NPCI.ORG.IN, renewer=yarn, realUser=, issueDate=1618831821344, maxDate=1619436621344, sequenceNumber=323, masterKeyId=30)] 2021-04-19 17:00:31,175 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. 2021-04-19 17:00:31,293 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data1/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data10/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data11/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data12/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data13/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data2/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data3/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data4/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data5/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data7/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data8/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009,/data9/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009 2021-04-19 17:00:31,740 INFO [main] org.apache.hadoop.mapred.YarnChild: /************************************************************ [system properties] os.name: Linux os.version: 3.10.0-1160.24.1.el7.x86_64 java.home: /usr/java/jdk1.8.0_281/jre java.runtime.version: 1.8.0_281-b09 java.vendor: Oracle Corporation java.version: 1.8.0_281 java.vm.name: Java HotSpot(TM) 64-Bit Server VM java.class.path: /data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002:/etc/hadoop/conf.cloudera.yarn:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/audience-annotations-0.5.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-util-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jul-to-slf4j-1.7.30.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jdom-1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-examples-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/log4j-api-2.8.2.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/flogger-system-backend-0.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jsr305-3.0.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/snakeyaml-1.26.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/spark-2.4.0.7.1.5.0-257-yarn-shuffle.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-lang3-3.8.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-math3-3.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/log4j-core-2.8.2.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/forbiddenapis-2.7.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-protobuf.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/ehcache-3.3.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerby-asn1-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/javax.servlet-api-3.1.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jakarta.xml.bind-api-2.3.2.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-annotations-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-identity-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-archives-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-pig-bundle.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-scala_2.10.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/checker-qual-2.8.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/aopalliance-1.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/stax2-api-3.1.4.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/slf4j-api-1.7.30.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/xz-1.8.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/groovy-json-3.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-xc-1.9.13.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-cli-1.2.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-extras-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/slf4j-log4j12.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/gson-2.2.4.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/fst-2.50.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-avro.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/tt-instrumentation-7.2.4.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jersey-client-1.19.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/lz4-1.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-core-asl-1.9.13-cloudera.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-admin-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-rbf-3.1.1.7.1.5.0-257-tests.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/dnsjava-2.1.7.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-module-jaxb-annotations-2.10.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/groovy-xml-3.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/javax.inject-1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerby-xdr-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-cascading3.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerby-config-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/mssql-jdbc-6.2.1.jre7.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jersey-core-1.19.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/flogger-0.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jettison-1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/asm-5.0.4.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/json-simple-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jakarta.activation-api-1.2.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/gateway-util-common.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-databind-2.10.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-hadoop.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-fs2img-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/ranger-yarn-plugin-shim-2.1.0.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-nfs-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-archive-logs-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/failureaccess-1.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jsr311-api-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-crypto-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-jaxrs-1.9.13.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerby-pkix-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-server-common-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/javax.activation-api-1.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/ranger-hdfs-plugin-shim-2.1.0.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/azure-keyvault-core-1.0.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-pig.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-xml-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/groovy-console-3.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/java-util-1.9.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/accessors-smart-1.2.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/aliyun-java-sdk-ram-3.0.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-core-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/nimbus-jose-jwt-7.9.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/logredactor-2.0.7.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/httpcore-4.4.10.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jersey-servlet-1.19.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-server-web-proxy-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-lang-2.6.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/curator-recipes-4.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-mapper-asl-1.9.13-cloudera.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/paranamer-2.8.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/json-smart-2.3.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jaxb-impl-2.2.3-1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/avro.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/groovy-templates-3.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-compress-1.19.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/woodstox-core-5.0.3.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-aliyun-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/guava-28.1-jre.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-common-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-rumen-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/httpclient-4.5.6.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-common-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/aliyun-java-sdk-ecs-4.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/groovy-3.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/protobuf-java-2.5.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-generator.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-applications-unmanaged-am-launcher-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-hadoop-bundle.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-azure-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-security-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-server-tests-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-hs-plugins-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jersey-server-1.19.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kafka-clients-0.8.2.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/ranger-raz-intg-2.1.0.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jersey-json-1.19.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-cascading.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/bcprov-jdk15on-1.60.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-core-2.10.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerby-util-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-api-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-datajoin-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-client-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-beanutils-1.9.4.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/javax.activation-1.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/gateway-shell.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-codec-1.11.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/groovy-swing-3.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-client-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jcip-annotations-1.0-1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-kafka-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-client-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-jobclient-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-nfs-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-native-client-3.1.1.7.1.5.0-257-tests.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-shuffle-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jersey-guice-1.19.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/zookeeper.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-auth-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/animal-sniffer-annotations-1.18.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-io-2.6.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-format-structures.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/ranger-plugin-classloader-2.1.0.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-ozone-filesystem-hadoop3-1.0.0.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-http-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/re2j-1.2.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-server-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-nativetask-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-servlet-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-sls-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-simplekdc-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/groovy-groovysh-3.0.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/zookeeper-jute.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-gridmix-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-common-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/j2objc-annotations-1.3.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-streaming-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-distcp-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-services-api-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-jackson.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-3.1.1.7.1.5.0-257-tests.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/okio-1.6.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-net-3.6.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-common-3.1.1.7.1.5.0-257-tests.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/log4j-1.2.17-cloudera1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/aliyun-sdk-oss-3.4.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-column.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/okhttp-2.7.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-configuration2-2.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-encoding.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jsch-0.1.54.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/snappy-java-1.1.7.7.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/curator-client-4.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-client-3.1.1.7.1.5.0-257-tests.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/guice-servlet-4.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/htrace-core4-4.1.0-incubating.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-common.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-kms-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jaxb-api-2.2.11.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/aliyun-java-sdk-core-3.4.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/metrics-core-3.2.4.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-annotations-2.10.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-hs-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/event-publish-7.2.4-shaded.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/google-extensions-0.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/aliyun-java-sdk-sts-3.0.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-common-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-logging-1.1.3.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-server-nodemanager-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-aws-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-registry-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/objenesis-1.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/zookeeper-3.5.5.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-uploader-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/checker-compat-qual-2.5.3.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/azure-data-lake-store-sdk-2.3.6.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/json-io-2.5.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/netty-all-4.1.51.Final.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-azure-datalake-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-util-ajax-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/zookeeper-jute-3.5.5.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/wildfly-openssl-1.0.7.Final.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-jaxrs-json-provider-2.10.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/parquet-thrift.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-configuration-1.10.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-openstack-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-webapp-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/azure-storage-7.0.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-httpfs-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-jobclient-3.1.1.7.1.5.0-257-tests.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-native-client-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-resourceestimator-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jackson-jaxrs-base-2.10.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/ranger-raz-hook-abfs-2.1.0.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/bcpkix-jdk15on-1.60.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-collections-3.2.2.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/HikariCP-java7-2.4.12.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/flogger-slf4j-backend-0.5.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/avro-1.8.2.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jna-5.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/swagger-annotations-1.5.4.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/kerb-server-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/commons-daemon-1.0.13.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/gcs-connector-2.1.2.7.1.5.0-257-shaded.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/token-provider-1.1.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-services-core-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-hdfs-rbf-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jsp-api-2.1.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/curator-framework-4.2.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/leveldbjni-all-1.8.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/gateway-i18n.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/aws-java-sdk-bundle-1.11.563.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/netty-3.10.6.Final.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/ojalgo-43.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-yarn-applications-distributedshell-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-util-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-core-3.1.1.7.1.5.0-257.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/guice-4.0.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/jetty-io-9.4.31.v20200723.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/gateway-cloud-bindings.jar:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/mr-framework/hadoop-mapreduce-client-app-3.1.1.7.1.5.0-257.jar::job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/job.jar java.io.tmpdir: /data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002/tmp user.dir: /data6/yarn/nm/usercache/megh.vidani/appcache/application_1618819500492_0009/container_e31_1618819500492_0009_01_000002 user.name: megh.vidani ************************************************************/ 2021-04-19 17:00:31,741 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 2021-04-19 17:00:32,042 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2 2021-04-19 17:00:32,042 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false 2021-04-19 17:00:32,056 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2021-04-19 17:00:32,235 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: /user/megh.vidani/.staging/_distcp-524677260/fileList.seq:0+500 2021-04-19 17:00:32,240 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2 2021-04-19 17:00:32,240 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false 2021-04-19 17:00:32,822 INFO [main] org.apache.hadoop.tools.mapred.CopyMapper: Copying webhdfs://<source-nameservice>/path/to/source/folder to webhdfs://<destination-nameservice>/path/to/source/folder 2021-04-19 17:00:32,915 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 0 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:00:32,972 INFO [main] org.apache.hadoop.tools.mapred.CopyMapper: Copying webhdfs://<source-nameservice>/path/to/source/folder/000031_0.gz to webhdfs://<destination-nameservice>/path/to/source/folder/000031_0.gz 2021-04-19 17:00:32,991 INFO [main] org.apache.hadoop.tools.mapred.RetriableFileCopyCommand: Copying webhdfs://<source-nameservice>/path/to/source/folder/000031_0.gz to webhdfs://<destination-nameservice>/path/to/source/folder/000031_0.gz 2021-04-19 17:00:32,991 INFO [main] org.apache.hadoop.tools.mapred.RetriableFileCopyCommand: Creating temp file: webhdfs://<destination-nameservice>/path/to/destination/folder/.distcp.tmp.attempt_1618819500492_0009_m_000000_0 2021-04-19 17:00:32,991 INFO [main] org.apache.hadoop.tools.mapred.RetriableFileCopyCommand: Writing to temporary target file path webhdfs://<destination-nameservice>/path/to/destination/folder/.distcp.tmp.attempt_1618819500492_0009_m_000000_0 2021-04-19 17:02:33,218 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode2-hostname>/<source-cluster-namenode2-ip>:50070. Already retried 0 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:02:33,305 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 1 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:03:33,350 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode2-hostname>/<source-cluster-namenode2-ip>:50070. Already retried 2 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 1441ms. 2021-04-19 17:03:34,874 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 3 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:04:34,916 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode2-hostname>/<source-cluster-namenode2-ip>:50070. Already retried 4 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 7259ms. 2021-04-19 17:04:42,257 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 5 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:05:42,349 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode2-hostname>/<source-cluster-namenode2-ip>:50070. Already retried 6 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 17414ms. 2021-04-19 17:05:59,845 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 7 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:06:59,916 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode2-hostname>/<source-cluster-namenode2-ip>:50070. Already retried 8 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 15922ms. 2021-04-19 17:07:15,921 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 9 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:08:16,013 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode2-hostname>/<source-cluster-namenode2-ip>:50070. Already retried 10 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 19586ms. 2021-04-19 17:08:35,740 ERROR [main] org.apache.hadoop.tools.util.RetriableCommand: Failure in Retriable command: Copying webhdfs://<source-nameservice>/path/to/source/folder/000031_0.gz to webhdfs://<destination-nameservice>/path/to/source/folder/000031_0.gz org.apache.hadoop.tools.mapred.RetriableFileCopyCommand$CopyReadException: org.apache.hadoop.ipc.StandbyException: Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.readBytes(RetriableFileCopyCommand.java:310) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.copyBytes(RetriableFileCopyCommand.java:256) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.copyToFile(RetriableFileCopyCommand.java:210) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:143) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:115) at org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87) at org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:258) at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:219) at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:48) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) Caused by: org.apache.hadoop.ipc.StandbyException: Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:110) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toIOException(WebHdfsFileSystem.java:561) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$900(WebHdfsFileSystem.java:144) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.shouldRetry(WebHdfsFileSystem.java:877) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:843) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:637) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:675) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:671) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$ReadRunner.read(WebHdfsFileSystem.java:2335) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$WebHdfsInputStream.read(WebHdfsFileSystem.java:2152) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.tools.util.ThrottledInputStream.read(ThrottledInputStream.java:91) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.readBytes(RetriableFileCopyCommand.java:308) ... 16 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error at org.apache.hadoop.hdfs.web.JsonUtilClient.toRemoteException(JsonUtilClient.java:90) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:522) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$300(WebHdfsFileSystem.java:144) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.connect(WebHdfsFileSystem.java:723) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$ReadRunner.connect(WebHdfsFileSystem.java:2386) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:814) ... 27 more 2021-04-19 17:08:37,073 INFO [main] org.apache.hadoop.tools.mapred.RetriableFileCopyCommand: Copying webhdfs://<source-nameservice>/path/to/source/folder/000031_0.gz to webhdfs://<destination-nameservice>/path/to/source/folder/000031_0.gz 2021-04-19 17:08:37,074 INFO [main] org.apache.hadoop.tools.mapred.RetriableFileCopyCommand: Creating temp file: webhdfs://<destination-nameservice>/path/to/destination/folder/.distcp.tmp.attempt_1618819500492_0009_m_000000_0 2021-04-19 17:08:37,074 INFO [main] org.apache.hadoop.tools.mapred.RetriableFileCopyCommand: Writing to temporary target file path webhdfs://<destination-nameservice>/path/to/destination/folder/.distcp.tmp.attempt_1618819500492_0009_m_000000_0 2021-04-19 17:08:37,094 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 0 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:10:37,248 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode2-hostname>/<source-cluster-namenode2-ip>:50070. Already retried 0 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms. 2021-04-19 17:10:37,333 INFO [main] org.apache.hadoop.hdfs.web.WebHdfsFileSystem: Retrying connect to namenode: <source-cluster-namenode1-hostname>/<source-cluster-namenode1-ip>:50070. Already retried 1 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@682c1e93, delay 0ms.       The same thing was working before enabling Kerberos.   Also, on the source cluster end, both active and standby namenode are working properly.   Can anybody point out any other place I should look into?   Thanks, Megh ... View more

Hello Everyone,   I'm facing a rather strange problem with my Ranger policies.   I'm not able to save modifications to any policies once they get created.   When I create a policy for the first time, it get created successfully. Subsequently, when I try to modify that policy, and click on save, the UI simply doesn't submit. Checked the Ranger Admin Logs as well, no entries.   Any suggestions?   Thanks, Megh ... View more

Hello Everyone,   Has anyone tested the Distcp editor available in HUE with CDP 7.1.5?   Currently I see the below screen:    In this, I'm not able to manually enter the source cluster path. As soon as I click on the source path text box, it opens a dialog box to select the path from the current cluster HDFS.   Has anybody tested pulling data from another cluster? What config changes are required for getting this to work?   Thanks, Megh ... View more