Skip to main content
FEIDAI
FEIDAI
Bronze III
June 21, 2021
Solved

HDFS 整个数据目录被删，请协助

HDFS服务  /user/hive/warehouse整个目录被删，请协助

 

 

amk
amkAnswer
Gold ⭐️
June 21, 2021

HDFS has a trash facility, in which deleted files are not actually deleted but rather are moved to a trash folder, where they remain for a minimum period before being permanently deleted by the system. The minimum period in minutes that a file will remain in the trash is set using the fs.trash.interval configuration property in core-site.xml.

 

The above said is true only when we have trash enabled or when -skipTrash is not used during delete operation. So please check whether the directory is present under "/user/<user-name>/.Trash". If the file is present then we can simply use mv or ditscp to recover the file to its original location.

 

If you have executed the operation recently then you can check NN UI for Blocks Pending Deletion. If this shows 0 blocks/relicas, then this means the blocks are already removed, and then we might need to use other mechanisms like snapshot or copy from the remote cluster. Please refer to below article for more details.


https://community.cloudera.com/t5/Internal/How-to-Recover-Files-in-HDFS-After-Removing-or-Deleting/ta-p/73885

View in thread

4 replies

VidyaSargur
Community Manager
VidyaSargur
Community Manager
June 21, 2021

Translation in English: The entire directory of HDFS service /user/hive/warehouse has been deleted, please assist

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
kingpin
Gold ⭐️
kingpin
Gold ⭐️
June 21, 2021

Hi @FEIDAI 

 

Check in hdfs trash if the deleted folder is there. (if you haven't used -skipTrash)

 

If you manage to find the folder under trash, copy it to your destination path

hdfs dfs -cp /user/hdfs/.Trash/Current/<your file> <destination>

 Otherwise, The best option is probably to find and use a data recovery tool or backup,

 

Regards,

 

 

Translate
amk
Gold ⭐️
amkAnswer
Gold ⭐️
June 21, 2021

HDFS has a trash facility, in which deleted files are not actually deleted but rather are moved to a trash folder, where they remain for a minimum period before being permanently deleted by the system. The minimum period in minutes that a file will remain in the trash is set using the fs.trash.interval configuration property in core-site.xml.

 

The above said is true only when we have trash enabled or when -skipTrash is not used during delete operation. So please check whether the directory is present under "/user/<user-name>/.Trash". If the file is present then we can simply use mv or ditscp to recover the file to its original location.

 

If you have executed the operation recently then you can check NN UI for Blocks Pending Deletion. If this shows 0 blocks/relicas, then this means the blocks are already removed, and then we might need to use other mechanisms like snapshot or copy from the remote cluster. Please refer to below article for more details.


https://community.cloudera.com/t5/Internal/How-to-Recover-Files-in-HDFS-After-Removing-or-Deleting/ta-p/73885

Translate
VidyaSargur
Community Manager
VidyaSargur
Community Manager
June 28, 2021

Hi @FEIDAI, has any of the replies resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
Terms & ConditionsAccessibility statement