Support Questions

Find answers, ask questions, and share your expertise

Creation of additional HDFS superuser

avatar
Contributor

Hi,

 

I am trying to create an additional HDFS useruser. So far, I have:

  1. Created a new user 
  2. Checked the value of Superuser Group (dfs.permissions.supergroup, dfs.permissions.superusergroup).
  3. Since we are using unix shell based groups mapping, I created a unix group on all hosts with a name matching the value of Superuser Group
  4. Added the desired superuser to the new group on all hosts.

I then tested and the new user cannot access all folders on HDFS.

 

A little investigation shows that neither dfs.permissions.superusergroup nor dfs.permissions.supergroup are present in either hdfs-site.xml or core-site.xml.

 

How do I make CM modify the configuration file so the superuser group is deployed.

 

Are there any other steps I am missing?

 

We are using CDH 5.16.

 

Thanks,

Tom

1 ACCEPTED SOLUTION

avatar

I assume you were checking in hdfs-site.xml of the /etc/hadoop/conf to validate the configuration change.

 

Through CM when you apply this change

CM > HDFS > Configuration > Superuser Group > Enter your desired supergroup name > Save and Restart

 

The change gets reflected in the server side [Namenodes, Datanodes] and it is not expected to be present in /etc/hadoop/conf directory. Because these properties are server side properties and hence not propagated to *-site.xml used by clients.

 

Just incase if you want to validate on the server side, you can search for this property in process directories where these services are running

grep "dfs.permissions.superusergroup" /var/run/cloudera-scm-agent/process/ -Rani

 

Additionally you can also make sure the group membership of the users by running

 

hdfs groups <supergroup-user>

 

View solution in original post

3 REPLIES 3

avatar
Contributor

I can see that after the group has been added, I need to do a dfsadmin -refreshUserToGroupsMappings

avatar

I assume you were checking in hdfs-site.xml of the /etc/hadoop/conf to validate the configuration change.

 

Through CM when you apply this change

CM > HDFS > Configuration > Superuser Group > Enter your desired supergroup name > Save and Restart

 

The change gets reflected in the server side [Namenodes, Datanodes] and it is not expected to be present in /etc/hadoop/conf directory. Because these properties are server side properties and hence not propagated to *-site.xml used by clients.

 

Just incase if you want to validate on the server side, you can search for this property in process directories where these services are running

grep "dfs.permissions.superusergroup" /var/run/cloudera-scm-agent/process/ -Rani

 

Additionally you can also make sure the group membership of the users by running

 

hdfs groups <supergroup-user>

 

avatar
Contributor

That command worked a treat, I wasn't aware that the configuration xmls varied based on the purpose of the host they are deployed to. 

 

I've got the superuser running now, thanks for the help