Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Highlighted

HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Hi All,

 

In our CDP 7.1.2 cluster, default ownership of any file or directory is <user>and supergroup.

One application team wants their data owner group to be their own group in stead of supergroup.

 

for example:

 

Let the app team user is 'abc' and group is 'abc'.

Currently owner and group for any file/dir in HDFS is abc:supergroup.

They want it to be abc:abc.

 

Please suggest how to achieve this so that any new file created by abc user will have ownership as abc:abc.

 

Regards,

Banshi. 

 

8 REPLIES 8
Highlighted

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Contributor

@banshidhar_saho 

 

This is the expected behavior as in Hadoop there are no groups based on user name. It does not use the OS level groups, instead it inherits it from parent directory. The reason is that group resolution happens on NameNode where groups related to the user may not exist.

 

To fix the case in your environment, modify the owner of the parent directory. This will cause new files to use the correct group.
Run:-->hdfs dfs -chown -R username:groupname <PATH to DIR>
This will recursively change the owner and group for the given <PATH>. (Note: do this with hdfs user)

 

You can further use acls' to ensure the groups you are trying have access to it.

 

hdfs dfs -setfacl -m default:group:<group-name>:rwx <Path>

 

Let me know if this helps

Highlighted

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Hi @tusharkathpal ,

 

Thank you very much. I tested it. It worked for ownership.

 

I want 750 permission on the directory recursively.

However it's not reflecting in case of new file.

 

Is there a way to do that? Idea is to restrict 'others' from accessing the file/dir.

 

Regards,

Banshi.

Highlighted

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Contributor

Hey @banshidhar_saho 

 

Glad to hear it works.

 

regarding permission, it depends upon acl permissions you give. In the previous comment I had updated rwx for group. You can set it something for "other"

 

hdfs dfs -setfacl -m default:group:<group-name>:rwx <Path>

 

hdfs dfs -setfacl -m default:other::- <path>

 

Can you try the above and update if it works.

Highlighted

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Mentor

@banshidhar_saho 

If you want to restrict the [others] you will need to apply POSIX permission on the file/directory look at the below examples

 

Below gives read and execute to [others]

 

$ hdfs dfs -setfacl -m others::r-x /path/to/dir

 

Below gives read to [others]

 

$ hdfs dfs -setfacl -m others::r-- /path/to/dir

 

Below gives no permissions to [others]

 

$ hdfs dfs -setfacl -m others::--- /path/to/dir

 

After the changes to validate run this snippet that should show it's set and to prove as [others]

 

$ hdfs dfs -getfacl /path/to/dir

 

 

try to list files in the directory 

Happy hadooping

Highlighted

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Hi @Shelton,

 

Thank you very much for sharing the exact commands.

 

Only thing that needs to change in the shared commands is - others should be other

Here is the command that worked for me.

 

$ hdfs dfs -setfacl -m other::--- /path/to/dir

 

Regards,

Banshi.

Highlighted

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Mentor

@banshidhar_saho 

 

Surely that was I typo error I am happy that it worked for you. If you feel that helped you solve the issue you have, it would be great to accept it as an answer so the thread can be marked a solution for other members to reference.

 

Happy hadooping

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Community Manager

Hi @banshidhar_saho , I'm happy to see you resolved your issue. Can you please mark the appropriate reply as the solution? It will surely make it easier for others to find the answer in the future.


Vidya Sargur, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Highlighted

Re: HDFS file/dir ownership to be <user>:<corresponding_group> in stead of <user>:supergroup

Hi Team,

 

Thank you very much for promptly responding and providing useful solution/direction.

 

While testing, app team reported that child dir/files are having r-x (for dir) and r--(files) for Others.

Looks like ACL are not getting inherited to child dir and files.

 

Can you confirm if ACL should inherit to child dir and files or not?

Is there any other configuration that I need to check and modify?

 

Regards,

Banshi.

Don't have an account?
Coming from Hortonworks? Activate your account here