Reply
Explorer
Posts: 9
Registered: ‎04-25-2018

User not returning any groups for hdfs groups <ID>

Hello,

 

I have one user ID which is not returning any groups for hdfs groups <ID>. However, groups <ID> is giving proper group mapping. Any thoughts? 

Posts: 1,673
Kudos: 329
Solutions: 263
Registered: ‎07-31-2013

Re: User not returning any groups for hdfs groups <ID>

Where are you executing this in your cluster?

The way 'hdfs groups' works is by sending an RPC request with the username to one of the NameNodes. When using the default ShellBasedUnixGroups plugin, the NameNode that received the request will run a 'id -gn username' command as a forked process on its own host and collect the output.

The key point here is that the groups check is not done on your host of invocation, as that'd be insecure to perform, it is done on the host of the service that is required to authorize a given request.

It is therefore critical that all hosts in the cluster report consistently the same group results for any given username. You can typically use a centralized identity management system with SSSD on Linux to achieve this (there are other ways too), instead of using local Linux /etc/passwd and /etc/group files to manage it (can get hairy to keep synced as the cluster grows).

For more behind the basics of auth(z), read http://blog.cloudera.com/blog/2012/03/authorization-and-authentication-in-hadoop/
Highlighted
Explorer
Posts: 9
Registered: ‎04-25-2018

Re: User not returning any groups for hdfs groups <ID>

Thanks, Harsh for your reply.

 

I am executing this from gateway node. I am using SSSD and able to fetch right groups using "groups <ID>" command. However, "hdfs groups" is not showing any groups. This is the same when checked from other nodes in the cluster as well. This is happening to only one particular user. 

Posts: 1,673
Kudos: 329
Solutions: 263
Registered: ‎07-31-2013

Re: User not returning any groups for hdfs groups <ID>

More specifically, what does 'groups username' report on all your NameNode
hosts?

Per the earlier post, the other hosts won't matter for a 'hdfs groups'
command check, only (all) your NameNode hosts' outputs would matter.

P.s. This is assuming you're using the shell based plugin in NameNode
configuration.
Explorer
Posts: 9
Registered: ‎04-25-2018

Re: User not returning any groups for hdfs groups <ID>

Hi,

 

I am getting the same outputs in my name nodes as well.

 

 

#groups <user ID>

Returns proper group mapping.

 

# hdfs groups <user ID>

No groups returned.

 

 

This is happening only for a specific user account and we are using ShellBasedUnixGroupsMapping. 

Sample log:

++++++++

org.apache.hadoop.security.ShellBasedUnixGroupsMapping: unable to return groups for user IPartialGroupNameException can't execute the shell command to get the list of group id for user 'ID' at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:228)

+++++++

 

 

Posts: 1,673
Kudos: 329
Solutions: 263
Registered: ‎07-31-2013

Re: User not returning any groups for hdfs groups <ID>

Thank you for confirming the verification over NameNode host(s).

The PartialGroupNameException will particularly trigger when the 'id -gn username && id -Gn username' returns some output but does not exit with a return code of 0. This is usually observed when the id command is unable to fully resolve all presented groups, which is likely what's happening.

- Do any of the outputs in the groups command you run return pure numeric results, instead of actual string names?
- What's the exit code after you execute 'id -gn username' for the affected user? You may run 'echo $?' to grab exit code after the command.
- Please paste the full stack trace, which should include a trace of an IOException after the log message as an underlying 'Caused by'. This would explain the reason behind why the partial group resolution further fails.
- Is there any particular difference to this username vs. others? For ex., does it start with a special character instead of alpha-num, etc.?
Announcements