Hello @wazzu62 , Kindly share what is the error message that you get while you run the hdfs balancer command as we haven't removed the CLI command on HDP 3.1.5.0.    ... View more

Hello @PrernaU ,   Unfortunately, ViewFS is not yet a supported feature on CDP as federation is not supported yet. ... View more

Looks like the fall back mechanism isn't been added    A fall back configuration is required at destination when running DistCP to copy files between a secure and an insecure cluster. Adding the following property to the advanced configuration snippet (if using Cloudera Manager) or of not, add it directly to the HDFS core-site.xml: <property> <name>ipc.client.fallback-to-simple-auth-allowed</name> <value>true</value> </property>   https://my.cloudera.com/knowledge/Copying-Files-from-Insecure-to-Secure-Cluster-using-DistCP?id=74873   ... View more

Ah ! got it .. thanks for update ! Yeah can you refer to the article once and can you also try to use copy from source namenode to destination namenode like this    hdfs://nn1:8020/foo/a hdfs://nn1:8020/foo/b https://hadoop.apache.org/docs/r3.0.3/hadoop-distcp/DistCp.html  ... View more

Can you please have a check if you have made the changes as per the below doc    https://docs.cloudera.com/cdp-private-cloud/latest/data-migration/topics/rm-migrate-securehdp-insecurecdp-distcp.html   As I see that you are migrating data from (Secured)HDP cluster to (unsecured)CDP cluster. Please correct me if my understanding is incorrect.    ... View more

Hi @vciampa ,   Looks like the arguments that are been passed is invalid Invalid arguments: Failed on local exception: java.io.IOException: java.io.EOFException; Host Details : local host is: "server2.localdomain/10.x.x.x"; destination host is: "svr1.local":9866;   Can you try to use source://nameservice:port dest://nameservice:port and try to run the distcp once. ... View more

Hi @enirys ,   From the below log snippet, I could think that the DN is running on the regular ports such as 50010 and 50075 (as per CM). Please confirm from your end.   2021-04-22 23:30:06,918 WARN conf.Configuration (Configuration.java:getTimeDurationHelper(1659)) - No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2021-04-22 23:30:06,924 ERROR datanode.DataNode (DataNode.java:secureMain(2692)) - Exception in secureMain java.lang.RuntimeException: Cannot start secure DataNode without configuring either privileged resources or SASL RPC data transfer protection and SSL for HTTP. Using privileged resources in combination with SASL RPC data transfer protection is not supported.   If they are running with the above ports are unprivileged ports on the OS. Could you try use the port 1024 (generally we do use 1004 and 1006)   Or the other option is to enable SASL RPC data transfer protection and TLS   The first option should be an easier one. Please try and let me know. ... View more

Hi @JGUI , There is no requirement for deleting the data from the datanode that is going to be decommissioned. Once the DN is been decommissioned all the blocks in the DN would be replicated to a different DN.   And is there any error that you are encountering while you are decommissioning ? Typically, HDFS would self-heal and would re-replicate the under-replicated blocks that are due to the DN that is been decommissioned. And NN would start replicating the blocks with the other two replication that is present in HDFS. ... View more