Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Solved Go to solution
Highlighted

Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Expert Contributor

Hello,

When i run hive commands, ranger audit is picking up my user name with Capitals e.g "John.Doe".

When I do HDFS Commands, its lower case "john.doe"

My Principal is: John.Doe@CORP.AD and we have auth-to-local rules to convert this to all lower case. (john.doe) In ranger we are also doing ranger.user.sync case conversion to lower so if we use user policies, only hdfs will work (e.g. i appear as john.doe in users and since Hive comes in as "John.Doe" user policies dont get applied). 

Example: CREATE TABLE test.permtest (field1 int); - the location of this folder is /data/2017

[john.doe@edge1 ~]$ hdfs dfs -ls /data/2017/

drwxr-xr-x - John.Doe hdfs 0 2017-05-02 20:43 /data/2017/permtest

As you can see from the above, the table gets created with the ACL permissions as John.Doe.

-------

Now when I do HDFS commands, e.g. it comes up as expected (john.doe - lower case)

[john.doe@edge1 ~]$ hdfs dfs -mkdir /data/2017/permtest1

drwxr-xr-x - John.Doe hdfs 0 2017-05-02 20:43 /data/2017/permtest

drwxr-xr-x - john.doe hdfs 0 2017-05-02 20:44 /data/2017/permtest1

The John.Doe and john.doe is what gets passed to ranger for authorization and this is a problem since user ranger sync brings over "john.doe" and so any Hive policies wont work.

Any ideas?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Expert Contributor

Look like it was just hive needed to be restarted (no restart prompt was there)...none of the above made any difference.

  1. ranger.usersync.ldap.username.caseconversion=lower
  2. ranger.usersync.ldap.groupname.caseconversion=lower

This is only used for usersync - how ranger imports your users in groups. It doesnt affect how your username or group will appear in audit

Please verify the auth_to_local rules in the host where hive is running, usually in /usr/hdp/<Version>/hadoop/conf

You can also try copy/link core-site.xml to /etc/hive/conf/conf.server and /etc/hive/conf

This didnt make any difference either, i believe its because it hive uses:

usr/hdp/current/hadoop-client/conf/:

5 REPLIES 5

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Master Collaborator

Can you try adding the property hadoop.security.auth_to_local to hive-site.xml as well, bounce the hive services and then try?

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Guru
@mliem

Have you tried this ?

ranger.usersync.ldap.username.caseconversion=lower
ranger.usersync.ldap.groupname.caseconversion=lower

then restart ranger.

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Super Collaborator
@mliem

Please verify the auth_to_local rules in the host where hive is running, usually in /usr/hdp/<Version>/hadoop/conf

You can also try copy/link core-site.xml to /etc/hive/conf/conf.server and /etc/hive/conf

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Expert Contributor

Look like it was just hive needed to be restarted (no restart prompt was there)...none of the above made any difference.

  1. ranger.usersync.ldap.username.caseconversion=lower
  2. ranger.usersync.ldap.groupname.caseconversion=lower

This is only used for usersync - how ranger imports your users in groups. It doesnt affect how your username or group will appear in audit

Please verify the auth_to_local rules in the host where hive is running, usually in /usr/hdp/<Version>/hadoop/conf

You can also try copy/link core-site.xml to /etc/hive/conf/conf.server and /etc/hive/conf

This didnt make any difference either, i believe its because it hive uses:

usr/hdp/current/hadoop-client/conf/:

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

Guru

Glad you were able to figure it out @mliem