Housekeeping of files under hdfs dfs command permission issue

[#1]

I'm trying to housekeep our hdfs filesystem and found alot of very old files that are owned by spark user. I don't remember creating a spark user that would generate these files, hence when I invoke an hdfs dfs -rm -R files...it's giving me a file ownership/permission error. 

Is there any work around I can bypass this? just like sudo in unix? Thank you.

[#2]

rm: Failed to move to trash: hdfs://aascmuatns1/data/farecard-batch-mgmt04/filelocations/part-r-00000-fffc9580-c310-44c2-9090-73ecf3133133: Permission denied: user=cdp_svc_fc_03, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

The above the error I'm getting, which I believe is expected.

[#3 · Accepted solution]

Hello @Grumash ,

I believe user=cdp_svc_fc_03 is the spark user which no longer exists. So when you are trying to move the file into the trash folder in the home dir, its failing to create the home dir. You need to create the home dir as the super user (hdfs), then chown it to cdp_svc_fc_03, then it should work.