But this still doesn't answer the original question - why is dfs.permissions.superusergroup defaulted to supergroup, and then CM doesn't create it in Linux? In our case, we also found discrepancies in other default Hadoop user/groups created from the documentation: Guide to Special Users in the Hadoop Environment For example, hdfs user not assigned to hdfs group, mapred group not created at all. We are running CDH 5.2.0 on Debian. I've done my share of research, and still find the HDFS user/group permission mechanism confusing. For the plain Linux-CDH installation without Kerberos (the majority I believe), HDFS relies on Unix user/group permission mechanism, but interprets in its own way. Thus the confusing and unintuitive behaviors: Unix root has less privilege than 'hdfs' user in HDFS - treated like an (uninitialized) regular user. No built-in user-admin commands for HDFS similar to Linux useradd, userdel, gpasswd, etc. No tool to 'migrate' existing Linux users to HDFS in bulk Hadoop app users that need to create HDFS files (e.g. mapred, flume, mapred, etc.) are not automatically set up in /user No pre-defined Unix group to include *all* Hadoop app users that need HDFS superuser access Regular users cannot run 'hdfs fsck /', since staging dir /tmp/logs/hdfs is 770 Perhaps Cloudera can write a more understandable adaptation of the Apache HDFS document. Thanks, Miles
... View more
