Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sentry integration with LDAP Groups

Sentry integration with LDAP Groups

New Contributor

In our environment, Sentry is not working with LDAP groups though Hadoop groups is successfully configured to work with LDAP groups

We are running CDH 4.3 and have configured Sentry for authorizing access with HiveServer2.  

"hdfs groups" command correctly shows all LDAP groups that the user is a member of. See below.

 

HiveServer2 log shows that Hadoop returned the OS groups to Sentry, not LDAP groups, even though Hadoop is configured to work with LDAP.

 

Appreciate any help / insights.  See relevant logs below:

 

 

Hive Server Log:

 

2014-04-08 10:04:31,033 DEBUG org.apache.sentry.provider.file.ResourceAuthorizationProvider: Authorization Request for Subject [name=tnj074] [Server [name=server1]] and [ALL]
2014-04-08 10:04:31,033 DEBUG org.apache.hadoop.security.Groups: Returning cached groups for 'tnj074'
2014-04-08 10:04:31,033 DEBUG org.apache.sentry.provider.file.SimplePolicyEngine: Getting permissions for [wheel] via null
2014-04-08 10:04:31,034 DEBUG org.apache.sentry.provider.file.Roles: Database null, Group wheel, Result []
2014-04-08 10:04:31,034 DEBUG org.apache.sentry.provider.file.SimplePolicyEngine: result = {}
2014-04-08 10:04:31,037 ERROR org.apache.hadoop.hive.ql.Driver: FAILED: SemanticException No valid privileges
org.apache.hadoop.hive.ql.parse.SemanticException: No valid privileges

 

[tnj074@d2phantd07 ~]$ hdfs groups
No encryption was performed by peer.
tnj074@COF.DS.CAPITALONE.COM : GR GG COF USR Mobile Users  GR_GG_COF_USR_HADOOPQADM GR_GG_COF_USR_HADOOPDADM GR GG COF SD WinZip v15.5 Win7 GR GG COF SD Microsoft Project Professional v2010 SP1
[tnj074@d2phantd07 ~]$

 

[tnj074@d2phantd07 ~]$ groups
wheel
[tnj074@d2phantd07 ~]$

 

[tnj074@d2phantd07 ~]$ hdfs dfs -cat /user/hive/sentry/sentry-provider.ini
No encryption was performed by peer.
[groups]
GR_GG_COF_USR_HADOOPQADM = role_bank_card_view
[roles]
all_server = server=server1
role_select = server=server1->db=default->table=test01
role_bank_card_view = server=server1
[tnj074@d2phantd07 ~]$

 

 

 

12 REPLIES 12

Re: Sentry integration with LDAP Groups

Master Guru
Ensure that your HiveServer2 service configuration directory's core-site.xml also carries the LDAP based groups class and configuration parameters, such as the ones configured at your NameNode.

Otherwise, the HS2 will employ defaults, which is a local group lookup.

Re: Sentry integration with LDAP Groups

Explorer

Hi Harsh J

We are stuck into same problem. Here are the summary

1. We have configured Sentry Service on Cloudera 5.3 (We have added "Sentry Service" not Policy file approach) . We have followed below reference URL
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_sg_sentry_service.ht...

2. Kerbros Authentication is not enabled on Cluster but as per prerequistee we can move ahead with LDAP Authentication also .
LDAP is configured on Cluster

3. After configuration, we go to beeline client and used “!connect jdbc:hive2://hadoopslave0.company.in:10000” as the connection string and entered “hive” as Username, Password
Here hiveserver2 is configured on hadoopslave0.company.in:10000 thats why we have given this in connection string & 1000 is default port.

After this when it ask to enter username & password so we have given "hive" in both (As per below URL
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/sg_sentry_overview.html...
To initiate top-level permissions for Sentry, an admin must login as a superuser that’s why we logged in as hive)

Now when we try to CREATE TABLE here so we are facing below error in this :-
Required privileges for this query: Server=server1->Db=default->action=*; (state=42000,code=40000)

Also error is coming when we try to give privilege to Groups (Group of LDAP in which LDAP user is member) .
GRANT ROLE qa TO GROUP TestGroup;
GRANT ALL ON DATABASE default TO ROLE qa WITH GRANT OPTION;

Problem Statement :- As we cant give permissions to LDAP groups and also cant create table so we are stucked to perform testing in Sentry enable environment.
It looks we are some how doing mistake in loggin with wrong user . We need to login with user who can give permission to other . We thought hive will work as superuser but it looks its not. If you can guide which user we should use to login to create table and GRANT privilege to other users so would be really helpful .

Kindly reply its very critical for us.

Re: Sentry integration with LDAP Groups

Master Guru
What is your groups lookup plugin set to? If its the default Shell Based mapping, then on both, the Sentry Service and the HS2 hosts, what is your output for the command "id -Gn hive && id -gn hive"?

Re: Sentry integration with LDAP Groups

Explorer

Hi Harsh,

 

First of all thank you very much for reply..Here are answers on your question

 

1. What is your groups lookup plugin set to ?

 

Ans :- I checked in our Cloudera manager console..... "org.apache.hadoop.security.LdapGroupsMapping" is set for  Hadoop User Group Mapping Implementation

 

2. What is your output for the command "id -Gn hive && id -gn hive"?

 

Ans :- I ran command "id -Gn hive && id -gn hive" on Sentry Host and Hive Server 2 host.. 

          Output on Hive Server 2 host :-  hive

                                                              hive

 

         Output on Sentry host :- hive hive-users

                                                  hive

 

 

Additional Question :- Is it mandatory to enable Kerbros authentication on Cluster before start work on Sentry ?

 

Thank you

Re: Sentry integration with LDAP Groups

Master Guru
Thanks, and what is the output also for "hdfs groups hive" command, since the LDAP mapping plugin is in use?

If you do not use Kerberos, you will need to run Sentry with the testing mode config set (sentry.hive.testing.mode set to true in sentry-site.xml valves).

Note that authorisation is also pointless without authentication. While you may be able to protect the HS2 end point, the HDFS data could still be exposed to abuse.

Re: Sentry integration with LDAP Groups

Explorer

Thanks Harsh

 

1. What is your output for the command "hdfs groups hive"?

 

Ans :- I ran command "hdfs groups hive" on Sentry Host and Hive Server 2 host.. 

          Output on Hive Server 2 host :- hive :

          Output on Sentry host            :- hive :

 

 

2. We have added below dependencies in sentry-site.xml in
/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/etc/sentry/conf.dist


<property>
<name>sentry.hive.testing.mode</name>
<value>true</value>
</property>

Re: Sentry integration with LDAP Groups

Explorer
Harsh can we Schedule a call to Discuss this problem.

Re: Sentry integration with LDAP Groups

Master Guru
If you are a Cloudera Support customer, please log a support case with the details and requests, to schedule a call to take this further.

Re: Sentry integration with LDAP Groups

Master Guru

It appears that your LDAP has no user "hive" and/or its not in its primary group "hive". This is why your Sentry and HS2 cannot promote "hive" as an automatic superuser.

 

We discourage use of the LdapGroupsMapping plugin generally; and if your shell is accessing the LDAP groups just fine (by means of the OS being configured to do so, like via SSSD or such), you can try to switch back the groups mapping plugin to the default Shell Based lookups to make things work.

Don't have an account?
Coming from Hortonworks? Activate your account here