HDFS User to Group Mapping

[#1]

On our clusters when a user creates an HDFS directory under /user/<username> the permissions are set as <username><username> instead of <username><user group>. 

We are using org.apache.hadoop.security.ShellBasedUnixGroupsMapping and we do have Kerberos enabled as well as LDAP authentication enabled for login. 

Is there a way to have the group ownership default to the user's group instead of the user name? 

[#2 · Accepted solution]

In HDFS, the permissions model for owner and group follow the BSD rule. The owner is set to the authenticated user, but the group is inherited from the parent directory. This is documented in the Permissions Guide: http://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.h...

"""
When a file or directory is created, its owner is the user identity of the client process, and its group is the group of the parent directory (the BSD rule).
"""

The Group Mapping is purely used at the authorisation side, not at the creation side as you are expecting it to be.

Since your /user/username directory's group is by default the username itself, that's the value you will naturally see for all groups. If you'd like that changed, you will need to chgrp the /user/username directory to be username:user-group instead of username:username. Subsequent files will now be created with username:user-group under it.