Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Housekeeping of files under hdfs dfs command permission issue

New Contributor

I'm trying to housekeep our hdfs filesystem and found alot of very old files that are owned by spark user. I don't remember creating a spark user that would generate these files, hence when I invoke an hdfs dfs -rm -R files...it's giving me a file ownership/permission error. 

Is there any work around I can bypass this? just like sudo in unix? Thank you.

 

1 ACCEPTED SOLUTION

Expert Contributor

Hello @Grumash ,

 

I believe user=cdp_svc_fc_03 is the spark user which no longer exists. So when you are trying to move the file into the trash folder in the home dir, its failing to create the home dir. You need to create the home dir as the super user (hdfs), then chown it to cdp_svc_fc_03, then it should work.

View solution in original post

2 REPLIES 2

New Contributor

rm: Failed to move to trash: hdfs://aascmuatns1/data/farecard-batch-mgmt04/filelocations/part-r-00000-fffc9580-c310-44c2-9090-73ecf3133133: Permission denied: user=cdp_svc_fc_03, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

 

The above the error I'm getting, which I believe is expected.

Expert Contributor

Hello @Grumash ,

 

I believe user=cdp_svc_fc_03 is the spark user which no longer exists. So when you are trying to move the file into the trash folder in the home dir, its failing to create the home dir. You need to create the home dir as the super user (hdfs), then chown it to cdp_svc_fc_03, then it should work.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.