question Re: Creation of additional HDFS superuser in Support Questions

Creation of additional HDFS superuser

ThomasHopewell — Mon, 02 Mar 2020 15:18:38 GMT

Hi,

I am trying to create an additional HDFS useruser. So far, I have:

Created a new user
Checked the value of Superuser Group (dfs.permissions.supergroup, dfs.permissions.superusergroup).
Since we are using unix shell based groups mapping, I created a unix group on all hosts with a name matching the value of Superuser Group
Added the desired superuser to the new group on all hosts.

I then tested and the new user cannot access all folders on HDFS.

A little investigation shows that neither dfs.permissions.superusergroup nor dfs.permissions.supergroup are present in either hdfs-site.xml or core-site.xml.

How do I make CM modify the configuration file so the superuser group is deployed.

Are there any other steps I am missing?

We are using CDH 5.16.

Thanks,

Tom

Re: Creation of additional HDFS superuser

ThomasHopewell — Mon, 02 Mar 2020 12:13:30 GMT

I can see that after the group has been added, I need to do a dfsadmin -refreshUserToGroupsMappings

Re: Creation of additional HDFS superuser

venkatsambath — Mon, 02 Mar 2020 14:30:29 GMT

I assume you were checking in hdfs-site.xml of the /etc/hadoop/conf to validate the configuration change.

Through CM when you apply this change

CM > HDFS > Configuration > Superuser Group > Enter your desired supergroup name > Save and Restart

The change gets reflected in the server side [Namenodes, Datanodes] and it is not expected to be present in /etc/hadoop/conf directory. Because these properties are server side properties and hence not propagated to *-site.xml used by clients.

Just incase if you want to validate on the server side, you can search for this property in process directories where these services are running

grep "dfs.permissions.superusergroup" /var/run/cloudera-scm-agent/process/ -Rani

Additionally you can also make sure the group membership of the users by running

hdfs groups <supergroup-user>

Re: Creation of additional HDFS superuser

ThomasHopewell — Wed, 04 Mar 2020 08:50:44 GMT

That command worked a treat, I wasn't aware that the configuration xmls varied based on the purpose of the host they are deployed to.

I've got the superuser running now, thanks for the help