Member since
03-29-2019
66
Posts
2
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1372 | 08-16-2023 09:33 AM | |
1669 | 06-21-2021 04:15 AM | |
1784 | 06-16-2021 01:08 AM | |
3740 | 05-02-2021 08:43 PM | |
1006 | 01-19-2020 08:07 AM |
08-16-2023
09:33 AM
Hi @skommineni We don't support setting Ozone as the default fs yet (Not fully integrated). This will be supported in future release. As a workaround you can try configuring HA (Active + Standby NN) instead of secondary NN and then set ozone as default fs.
... View more
06-21-2021
04:15 AM
HDFS has a trash facility, in which deleted files are not actually deleted but rather are moved to a trash folder, where they remain for a minimum period before being permanently deleted by the system. The minimum period in minutes that a file will remain in the trash is set using the fs.trash.interval configuration property in core-site.xml. The above said is true only when we have trash enabled or when -skipTrash is not used during delete operation. So please check whether the directory is present under "/user/<user-name>/.Trash". If the file is present then we can simply use mv or ditscp to recover the file to its original location. If you have executed the operation recently then you can check NN UI for Blocks Pending Deletion. If this shows 0 blocks/relicas, then this means the blocks are already removed, and then we might need to use other mechanisms like snapshot or copy from the remote cluster. Please refer to below article for more details. https://community.cloudera.com/t5/Internal/How-to-Recover-Files-in-HDFS-After-Removing-or-Deleting/ta-p/73885
... View more
06-17-2021
09:24 PM
Can you please post the error received and directory usage along with the quota configured for the user?
... View more
06-16-2021
01:08 AM
1 Kudo
Hello @pauljoshiva The NameNode endeavors to ensure that each block always has the intended number of replicas. The NameNode detects that a block has become under- or over-replicated when a block report from a DataNode arrives. When a block becomes over replicated, the NameNode chooses a replica to remove. The NameNode will prefer not to reduce the number of racks that host replicas, and secondly prefer to remove a replica from the DataNode with the least amount of available disk space. The goal is to balance storage utilization across DataNodes without reducing the block's availability. Hope this answers your query. Regards, Manoj
... View more
05-02-2021
08:43 PM
Yes, right. Please move all the contents and start RMAN service once again.
... View more
04-30-2021
05:14 AM
1 Kudo
Hello Amn, Thanks for using Cloudera Community. I believe this issue is happening due to index failure. Can you please follow the below steps? 1. Stop the RMAN service from CM. 2. Navigate to CM > Cloudera Management Service > Configuration and search for Reports Manager Working Directory By default it would be /var/lib/cloudera-scm-headlamp Once identified we can move contents of /var/lib/cloudera-scm-headlamp/HDFS-1/ns1/* to some other tmp directory. Post that start RMAN service. This should help us in rebuilding the index once again. Please let me know if it helps.
... View more
03-14-2021
09:02 PM
We are seeing port bind exception in the error stack trace which basically means the secondary NN service is unable to register itself on that port. Port 50090 is the internal port defined by this property "dfs.secondary.http.address or dfs.namenode. secondary. http-address". So please run "netstat -anp | grep 50090" and see which process is using the port in question. Stop that process and try starting Secondary NN service or else we need to change the default port in the abovementioned property to some other non-used port.
... View more
03-11-2021
09:14 PM
Apart from the command line, we can view this information from Cloudera Manager. Please Navigate to Cloudera Manager > HDFS > File Browser and click Directory you are interested to see quota usage. Also for a consolidated report did you try checking disk usage reports available in CM? We can download the usage report in CSV format for offline analysis. Please refer to below link for more details. https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_directory_usage.html
... View more
01-19-2020
08:07 AM
How much data did you delete ? Did checkpoint happen after you deleted data ? Also please check if any snapshots are present ? HDFS CLI "du" output not only include normal files but also includes the files that have been deleted and exist in snapshots (which is true in terms of real resource consumption). Please check the output using -x flag which excludes snapshot from calculation. hdfs dfs -du -x -s -h /path
... View more