Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
cleanup hdfs files owned by a user
Labels:
- Labels:
-
Apache Hadoop
Super Collaborator
Created ‎01-02-2018 09:29 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I want to find files owned by a user in HDFS and delete all of them at once.
1 REPLY 1
Super Guru
Created ‎01-02-2018 11:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The below command should do
hdfs dfs -ls -R / | awk '{ if ( $3 == "spark" && substr($0,0,1) != "d" ) { print $8 } }' | xargs hdfs dfs -rm
In the above command "spark" is the user name. Replace it with your username. Also I considered path as '/' . If you want to delete files only in a certain directory , replace it with your directory.
This will remove only the files owned by the user and not the directories.
Thanks,
Aditya
