HDP 2.4 - How to create Kerberos token for non-service user (hbase_user1)

[#1]

hi All, i've a Kerberized HDP 2.4 cluster, and i've a user created - hbase_user1

To create a kerberos token, i do the following ->

----------------------------------

[hbase_user1@sandbox ~]$ klist Ticket cache: FILE:/tmp/krb5cc_1029 Default principal: hbase_user1@EXAMPLE.COM Valid starting Expires Service principal 01/31/17 21:44:20 02/01/17 21:44:20 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 01/31/17 21:44:20

----------------------------------

Is this the correct way to create the kerberos token, so user can access Hbase tables ?

when i try to use headless kerberos, it gives error as shown below ->

---------------------------------------------------------------------------------------

[hbase_user1@sandbox ~]$ klist -kte /etc/security/keytabs/hbase.headless.keytab Keytab name: FILE:/etc/security/keytabs/hbase.headless.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (arcfour-hmac) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (des-cbc-md5) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (des3-cbc-sha1) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (aes256-cts-hmac-sha1-96) [hbase_user1@sandbox ~]$ kinit -k -t /etc/security/keytabs/hbase.headless.keytab hbase-Sandbox@EXAMPLE.COM kinit: Password incorrect while getting initial credentials

------------------------------------------------------------------------------------------

[#2]

@Alex Miller - any ideas on this ?

[#3]

It's not clear why kinit doesn't work. You may try to get log from Kerberos client using KRB5_TRACE env variable:

KRB5_TRACE=/tmp/log kinit -k -t /etc/security/keytabs/hbase.headless.keytab hbase-Sandbox@EXAMPLE.COM

Usually it helps to identify the problem. Most common problems: principal was modified after keytab creation (so you need to regenerate keytab), keytab was created without -norandkey option (but usually in this case kinit with password would not work). But I would suggest avoid using service tickets, but grant permissions to the hbase_user1 for tables it need to access. Use hbase shell to do that

grant 'hbase_user1', 'RWCA',  'table_name'

[#4]

@Sergey Soldatov - i ran the following command as suggested, here is the output of the

KRB5_TRACE=/tmp/log kinit -k -t /etc/security/keytabs/hbase.headless.keytab hbase-Sandbox@EXAMPLE.COM

-----------------------------

[hbase_user1@sandbox ~]$ cat /tmp/log [26961] 1485902434.555523: Getting initial credentials for hbase-Sandbox@EXAMPLE.COM [26961] 1485902434.555794: Looked up etypes in keytab: aes128-cts, rc4-hmac, des, des-cbc-crc, des3-cbc-sha1, aes256-cts [26961] 1485902434.555828: Sending request (205 bytes) to EXAMPLE.COM [26961] 1485902434.555882: Resolving hostname sandbox.hortonworks.com [26961] 1485902434.556115: Sending initial UDP request to dgram 10.0.2.15:88 [26961] 1485902434.556452: Received answer from dgram 10.0.2.15:88 [26961] 1485902434.556473: Response was not from master KDC [26961] 1485902434.556502: Processing preauth types: 19 [26961] 1485902434.556512: Selected etype info: etype aes256-cts, salt "(null)", params "" [26961] 1485902434.556516: Produced preauth for next request: (empty) [26961] 1485902434.556521: Salt derived from principal: EXAMPLE.COMhbase-Sandbox [26961] 1485902434.556525: Getting AS key, salt "EXAMPLE.COMhbase-Sandbox", params "" [26961] 1485902434.556569: Retrieving hbase-Sandbox@EXAMPLE.COM from FILE:/etc/security/keytabs/hbase.headless.keytab (vno 0, enctype aes256-cts) with result: 0/Success [26961] 1485902434.556590: AS key obtained from gak_fct: aes256-cts/FB0A [26961] 1485902434.556634: Getting initial credentials for hbase-Sandbox@EXAMPLE.COM [26961] 1485902434.556688: Looked up etypes in keytab: aes128-cts, rc4-hmac, des, des-cbc-crc, des3-cbc-sha1, aes256-cts [26961] 1485902434.556705: Sending request (205 bytes) to EXAMPLE.COM (master)

-----------------------------

wrt granting access to hbase table, i've already done that using Ranger, but still facing the issue... this is part of the debugging for that access issue.

Pls see link to issue created for the access issue ->

https://community.hortonworks.com/questions/80406/hdp24-hbase-permissions-using-ranger-not-working.h...

Any ideas ?

[#5]

@Sergey Soldatov -

thanks, granting access to the user - hbase_user1 actually worked.

However, i'd already given access to the table using Apache Ranger. Shouldn't that have worked ?

What needs to be done to check/ensuure the Ranger integration with Hbase to control access ?

A few more details -

Actually, when i check the Ranger audit logs - the user 'hbase_user1' appears only till December 2nd, 2017 - screenshot

screen-shot-2017-01-31-at-60941-pm.png

This setup was created in Nov, and was working when i checked last year,

Attaching screenshots of the ranger policies. - global access to hbase is removed, and access to table - 'iemployee' is provided to user - hbase_user1.

screen-shot-2017-01-31-at-51919-pm.png

screen-shot-2017-01-31-at-51817-pm.png

[#6]

@Karan Alang

Did you use password while creating user "hbase_user1" Keytab in LDAP AD?

if yes, Did you copy the keytab to /etc/security/keytab/ location?

if yes, please run below command to validate your keytab

cd /etc/security/keytab/

klist -kte hbase.headless.keytab

if you have correct key you should able to see the hostname details.

-----------------------------

Here is the ktpass sample command for create keytab in LDAP AD

ktpass -out hbase_user1.service.keytab -princ hbase-Sandbox@EXAMPLE.COM -mapuser "admin" -mapop set -ptype KRB5_NT_PRINCIPAL -crypto All -pass *

[#7]

@Divakar Annapureddy - yes, had created the password for hbase_user1, and am able to see the details of the keytab, when logged in as hbase_user1

---------------------------------------------------------------------------------------------------------

[hbase_user1@sandbox ~]$ klist -kte /etc/security/keytabs/hbase.headless.keytab

Keytab name: FILE:/etc/security/keytabs/hbase.headless.keytab

KVNO Timestamp Principal

---- ----------------- --------------------------------------------------------

1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (aes128-cts-hmac-sha1-96)

1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (arcfour-hmac)

1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (des-cbc-md5)

1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (des3-cbc-sha1)

1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (aes256-cts-hmac-sha1-96)

However, as i mentioned, when i fire the following command i get error -

-----------------------------------

kinit -k -t /etc/security/keytabs/hbase.headless.keytab hbase-Sandbox@EXAMPLE.COM

kinit: Password incorrect while getting initial credentials

-----------------------------------

So, what needs to be done to fix this ?

Alternately, i fire command which goes through ->

kinit hbase_user1@EXAMPLE.COM

But it is not allowing me to access the tables, even though access is given to the user using Ranger.

[#8]

@Karan Alang

please try to run below commands and give me the output?

1) klist -k -t -e /etc/security/keytabs/hbase.headless.keytab

2) kinit -kt hbase.headless.keytab hbase-Sandbox@EXAMPLE.COM

3) klist

[#9]

@Divakar Annapureddy - pls see below.

[hbase_user1@sandbox ~]$ klist -k -t -e /etc/security/keytabs/hbase.headless.keytab Keytab name: FILE:/etc/security/keytabs/hbase.headless.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (arcfour-hmac) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (des-cbc-md5) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (des3-cbc-sha1) 1 11/29/16 00:19:50 hbase-Sandbox@EXAMPLE.COM (aes256-cts-hmac-sha1-96)

[hbase_user1@sandbox ~]$ kinit -kt /etc/security/keytabs/hbase.headless.keytab hbase-Sandbox@EXAMPLE.COM kinit: Password incorrect while getting initial credentials

Currently, i've the following kerberos token for hbase_user1 ->

-------------------------------------

[hbase_user1@sandbox ~]$ klist Ticket cache: FILE:/tmp/krb5cc_1029 Default principal: hbase_user1@EXAMPLE.COM Valid starting Expires Service principal 02/01/17 01:15:27 02/02/17 01:15:27 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 02/01/17 01:15:27

------------------------------------

However, it seems the Actual issue i'm trying to solve is not because kerberos, somehow permission on HBase tables put in Ranger are not taking effect.

When i put int access permissions on commandline (with existing kerberos), i'm able to restrict/provide access.

Pls see the link below -

https://community.hortonworks.com/questions/80798/kerberized-hdp-24-hbase-user-not-able-to-access-ta...