question Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working) in Archives of Support Questions (Read Only)

Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

mliem — Wed, 03 May 2017 10:46:00 GMT

Hello,

When i run hive commands, ranger audit is picking up my user name with Capitals e.g "John.Doe".

When I do HDFS Commands, its lower case "john.doe"

My Principal is: John.Doe@CORP.AD and we have auth-to-local rules to convert this to all lower case. (john.doe) In ranger we are also doing ranger.user.sync case conversion to lower so if we use user policies, only hdfs will work (e.g. i appear as john.doe in users and since Hive comes in as "John.Doe" user policies dont get applied).

Example: CREATE TABLE test.permtest (field1 int); - the location of this folder is /data/2017

[john.doe@edge1 ~]$ hdfs dfs -ls /data/2017/

drwxr-xr-x - John.Doe hdfs 0 2017-05-02 20:43 /data/2017/permtest

As you can see from the above, the table gets created with the ACL permissions as John.Doe.

-------

Now when I do HDFS commands, e.g. it comes up as expected (john.doe - lower case)

[john.doe@edge1 ~]$ hdfs dfs -mkdir /data/2017/permtest1

drwxr-xr-x - John.Doe hdfs 0 2017-05-02 20:43 /data/2017/permtest

drwxr-xr-x - john.doe hdfs 0 2017-05-02 20:44 /data/2017/permtest1

The John.Doe and john.doe is what gets passed to ranger for authorization and this is a problem since user ranger sync brings over "john.doe" and so any Hive policies wont work.

Any ideas?

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

deepesh1 — Wed, 03 May 2017 11:06:24 GMT

Can you try adding the property hadoop.security.auth_to_local to hive-site.xml as well, bounce the hive services and then try?

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

srai1 — Wed, 03 May 2017 11:30:11 GMT

@mliem

Have you tried this ?

ranger.usersync.ldap.username.caseconversion=lower
ranger.usersync.ldap.groupname.caseconversion=lower

then restart ranger.

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

rguruvannagari — Wed, 03 May 2017 11:41:30 GMT

@mliem

Please verify the auth_to_local rules in the host where hive is running, usually in /usr/hdp/<Version>/hadoop/conf

You can also try copy/link core-site.xml to /etc/hive/conf/conf.server and /etc/hive/conf

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

mliem — Wed, 03 May 2017 21:25:39 GMT

Look like it was just hive needed to be restarted (no restart prompt was there)...none of the above made any difference.

ranger.usersync.ldap.username.caseconversion=lower
ranger.usersync.ldap.groupname.caseconversion=lower

This is only used for usersync - how ranger imports your users in groups. It doesnt affect how your username or group will appear in audit

Please verify the auth_to_local rules in the host where hive is running, usually in /usr/hdp/<Version>/hadoop/conf

You can also try copy/link core-site.xml to /etc/hive/conf/conf.server and /etc/hive/conf

This didnt make any difference either, i believe its because it hive uses:

usr/hdp/current/hadoop-client/conf/:

Re: Hive not applying auth_to_local rules with Kerberos (HDFS commands working)

srai1 — Fri, 05 May 2017 02:28:50 GMT

Glad you were able to figure it out @mliem