How to Manage HDFS User Groups: Why "usermod -G" Must Be Run on the NameNode

Question

Hello,   I  want to do a folder that can be accessed in r, w,x by all the users of the group 'developer' that I created.  I also want that all the folders/files created by these users in this folder can be accessed in r,w,x by the users from this group.  All the other users should access the folder in r and x.   What should I do?   I tried:   - to create my folder with these rights: drwxrwxr-t+   - the folder that I created is owned by admin who is part of the developer group.   All I think is missing is to add the default group of my users the developer group : do something like hadoop def -usermod -G developer Peter But this is not working!      Thank you!

denloe · Accepted Answer

Creating a user in Hue creates the user only in the Hue User database.

Creating a user in Cloudera Manager create the user only the the Cloudera
Manager User table.

Both the user and the groups need to exist on the NameNode host operating
system:

sudo useradd Peter
sudo usermod -G developer Peter

If you don't want the user to be able to log in to the NameNode:

sudo usermod -s /bin/falso Peter
or
sudo usermod -s /usr/bin/nologin Peter

denloe · Answer

To begin with, my explanation will cover the case that you are using simple
authentication and the default Hadoop group mappings.

Hadoop does not use uid or gid.

You username on the system is passed to the namenode when you make a client
call.

hadoop fs -ls /

If your username is 'bob', the NameNode will receive the string "bob" to
identify you.
If your username is 'hdfs', the NameNode will receive the string "hdfs" to
identify you.

User and Group lookups are done by the NameNode.

The default for the property hadoop.security.group.mapping
is org.apache.hadoop.security.ShellBasedUnixGroupsMapping, which means a
username passed with an HDFS client API will do the equivalent of "id -Gn
" on the NameNode host.

On the host running the NameNode, use the standard unix commands to assign
users to groups.

$ sudo usermod -G developer Peter

You only need to define the group membership on the NameNode's host
operating system. Now if a user named 'Peter' exists on any other system
configured as an HDFS client, they will still be considered part of the
"developer" group by HDFS.

"hdfs groups " is a handy command you can run to see if a user
belongs to a group in HDFS

Create your directory in HDFS:

hadoop fs -mkdir /projects/only/developers
hadoop fs -chown anyuser:developers /projects/only/developers
hadoop fs -chmod 775 /projects/only/developers

Your directory is now writable by anyone in the group "developers" as
defined on the NameNode.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.