question Housekeeping of files under hdfs dfs command permission issue in Support Questions

Housekeeping of files under hdfs dfs command permission issue

Grumash — Tue, 21 Apr 2026 07:54:26 GMT

I'm trying to housekeep our hdfs filesystem and found alot of very old files that are owned by spark user. I don't remember creating a spark user that would generate these files, hence when I invoke an hdfs dfs -rm -R files...it's giving me a file ownership/permission error.

Is there any work around I can bypass this? just like sudo in unix? Thank you.

Re: Housekeeping of files under hdfs dfs command permission issue

Grumash — Wed, 22 Jun 2022 09:18:14 GMT

rm: Failed to move to trash: hdfs://aascmuatns1/data/farecard-batch-mgmt04/filelocations/part-r-00000-fffc9580-c310-44c2-9090-73ecf3133133: Permission denied: user=cdp_svc_fc_03, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

The above the error I'm getting, which I believe is expected.

Re: Housekeeping of files under hdfs dfs command permission issue

rki_ — Thu, 30 Jun 2022 10:41:58 GMT

Hello @Grumash ,

I believe user=cdp_svc_fc_03 is the spark user which no longer exists. So when you are trying to move the file into the trash folder in the home dir, its failing to create the home dir. You need to create the home dir as the super user (hdfs), then chown it to cdp_svc_fc_03, then it should work.