Creation of additional HDFS superuser

Question

Hi,

I am trying to create an additional HDFS useruser. So far, I have:

Created a new user
Checked the value of Superuser Group (dfs.permissions.supergroup, dfs.permissions.superusergroup).
Since we are using unix shell based groups mapping, I created a unix group on all hosts with a name matching the value of Superuser Group
Added the desired superuser to the new group on all hosts.

I then tested and the new user cannot access all folders on HDFS.

A little investigation shows that neither dfs.permissions.superusergroup nor dfs.permissions.supergroup are present in either hdfs-site.xml or core-site.xml.

How do I make CM modify the configuration file so the superuser group is deployed.

Are there any other steps I am missing?

We are using CDH 5.16.

Thanks,

Tom

venkatsambath · Accepted Answer

I assume you were checking in hdfs-site.xml of the /etc/hadoop/conf to validate the configuration change.

Through CM when you apply this change

CM > HDFS > Configuration > Superuser Group > Enter your desired supergroup name > Save and Restart

The change gets reflected in the server side [Namenodes, Datanodes] and it is not expected to be present in /etc/hadoop/conf directory. Because these properties are server side properties and hence not propagated to *-site.xml used by clients.

Just incase if you want to validate on the server side, you can search for this property in process directories where these services are running

grep "dfs.permissions.superusergroup" /var/run/cloudera-scm-agent/process/ -Rani

Additionally you can also make sure the group membership of the users by running

hdfs groups <supergroup-user>

ThomasHopewell · Answer

I can see that after the group has been added, I need to do a dfsadmin -refreshUserToGroupsMappings

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.