Creation of additional HDFS superuser

[#1]

Hi,

I am trying to create an additional HDFS useruser. So far, I have:

1. Created a new user 
2. Checked the value of Superuser Group (dfs.permissions.supergroup, dfs.permissions.superusergroup).
3. Since we are using unix shell based groups mapping, I created a unix group on all hosts with a name matching the value of Superuser Group
4. Added the desired superuser to the new group on all hosts.

I then tested and the new user cannot access all folders on HDFS.

A little investigation shows that neither dfs.permissions.superusergroup nor dfs.permissions.supergroup are present in either hdfs-site.xml or core-site.xml.

How do I make CM modify the configuration file so the superuser group is deployed.

Are there any other steps I am missing?

We are using CDH 5.16.

Thanks,

Tom

[#2]

I can see that after the group has been added, I need to do a dfsadmin -refreshUserToGroupsMappings

[#3 · Accepted solution]

I assume you were checking in hdfs-site.xml of the /etc/hadoop/conf to validate the configuration change.

Through CM when you apply this change

CM > HDFS > Configuration > Superuser Group > Enter your desired supergroup name > Save and Restart

The change gets reflected in the server side [Namenodes, Datanodes] and it is not expected to be present in /etc/hadoop/conf directory. Because these properties are server side properties and hence not propagated to *-site.xml used by clients.

Just incase if you want to validate on the server side, you can search for this property in process directories where these services are running

grep "dfs.permissions.superusergroup" /var/run/cloudera-scm-agent/process/ -Rani

Additionally you can also make sure the group membership of the users by running

hdfs groups <supergroup-user>

[#4]

That command worked a treat, I wasn't aware that the configuration xmls varied based on the purpose of the host they are deployed to. 

I've got the superuser running now, thanks for the help