Support Questions

Find answers, ask questions, and share your expertise

How can zk client or hdfs client connect to server which on another kerberos realm

Explorer

I had already configure kerberos cross realm trust , 

here is my kvno

 

$ kinit e3base/xardc2@QINRC_REALM.COM

 

$ kvno e3base/kf-app82@e3base_kfapp
e3base/kf-app82@e3base_kfapp: kvno = 2

 

$ klist
Ticket cache: FILE:/tmp/krb5cc_1102
Default principal: e3base/xardc2@QINRC_REALM.COM

Valid starting Expires Service principal
05/15/18 10:26:59 05/15/18 10:56:59 krbtgt/QINRC_REALM.COM@QINRC_REALM.COM
renew until 05/15/18 10:56:59
05/15/18 10:51:58 05/15/18 10:56:59 krbtgt/e3base_kfapp@QINRC_REALM.COM
renew until 05/15/18 10:56:59
05/15/18 10:52:37 05/15/18 10:56:59 e3base/xardc2@e3base_kfapp
renew until 05/15/18 10:56:59
05/15/18 10:55:32 05/15/18 10:56:59 e3base/kf-app82@e3base_kfapp
renew until 05/15/18 10:56:59

 

 

so it seems that corss realm trust is work .

 

but when i use zkCli on realm QINRC_REALM.COM to connect zkerver which on realm e3base_kfapp

 

AuthFailed

 

Debug log:

$ zkCli.sh -server kfapp74:11001
Connecting to kfapp74:11001
Welcome to ZooKeeper!
JLine support is enabled
>>> KeyTabInputStream, readName(): QINRC_REALM.COM
>>> KeyTabInputStream, readName(): e3base
>>> KeyTab: load() entry length: 56; type: 23
>>> KeyTabInputStream, readName(): QINRC_REALM.COM
>>> KeyTabInputStream, readName(): e3base
>>> KeyTab: load() entry length: 48; type: 8
>>> KeyTabInputStream, readName(): QINRC_REALM.COM
>>> KeyTabInputStream, readName(): e3base
>>> KeyTab: load() entry length: 48; type: 3
Looking for keys for: e3base@QINRC_REALM.COM
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
Found unsupported keytype (3) for e3base@QINRC_REALM.COM
Found unsupported keytype (8) for e3base@QINRC_REALM.COM
Added key: 23version: 2
>>> KdcAccessibility: reset
Looking for keys for: e3base@QINRC_REALM.COM
Found unsupported keytype (3) for e3base@QINRC_REALM.COM
Found unsupported keytype (8) for e3base@QINRC_REALM.COM
Added key: 23version: 2
default etypes for default_tkt_enctypes: 23.
>>> KrbAsReq creating message
[zk: kfapp74:11001(CONNECTING) 0] >>> KrbKdcReq send: kdc=xardc2 UDP:21732, timeout=2500, number of retries =3, #bytes=141
>>> KDCCommunication: kdc=xardc2 UDP:21732, timeout=2500,Attempt =1, #bytes=141
>>> KrbKdcReq send: #bytes read=641
>>> KdcAccessibility: remove xardc2:21732
Looking for keys for: e3base@QINRC_REALM.COM
Found unsupported keytype (3) for e3base@QINRC_REALM.COM
Found unsupported keytype (8) for e3base@QINRC_REALM.COM
Added key: 23version: 2
>>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
>>> KrbAsRep cons in KrbAsReq.getReply e3base

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
Found ticket for e3base@QINRC_REALM.COM to go to krbtgt/QINRC_REALM.COM@QINRC_REALM.COM expiring on Tue May 15 11:48:57 CST 2018
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for e3base@QINRC_REALM.COM to go to krbtgt/QINRC_REALM.COM@QINRC_REALM.COM expiring on Tue May 15 11:48:57 CST 2018
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
default etypes for default_tgs_enctypes: 23.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
>>> KrbKdcReq send: kdc=xardc2 UDP:21732, timeout=2500, number of retries =3, #bytes=635
>>> KDCCommunication: kdc=xardc2 UDP:21732, timeout=2500,Attempt =1, #bytes=635
>>> KrbKdcReq send: #bytes read=176
>>> KdcAccessibility: remove xardc2:21732
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
cTime is Mon Sep 29 21:28:06 CST 1975 181229286000
sTime is Tue May 15 11:18:57 CST 2018 1526354337000
suSec is 776245
error code is 7
error Message is Server not found in Kerberos database
cname is e3base@QINRC_REALM.COM
sname is zookeeper/kfapp74@QINRC_REALM.COM
msgType is 30
KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270)
at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302)
at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120)
at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
at org.apache.zookeeper.client.ZooKeeperSaslClient$1.run(ZooKeeperSaslClient.java:283)
at org.apache.zookeeper.client.ZooKeeperSaslClient$1.run(ZooKeeperSaslClient.java:280)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:279)
at org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:265)
at org.apache.zookeeper.client.ZooKeeperSaslClient.sendSaslPacket(ZooKeeperSaslClient.java:337)
at org.apache.zookeeper.client.ZooKeeperSaslClient.initialize(ZooKeeperSaslClient.java:375)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1013)
Caused by: KrbException: Identifier doesn't match expected value (906)
at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140)
at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65)
at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60)
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55)
... 18 more

 

From the log  i can know that client princle is e3base@QINRC_REALM.COM, and server principle is zookeeper/kfapp74@QINRC_REALM.COM, but why can zkCli find server principle on realm e3base_kfapp? because zkServer on host kfapp74 belongs to realm e3base_kfapp , and zkCli belongs to realm QINRC_REALM.COM.

 

how can i do to let zkCli find server princle from realm e3base_kfapp ? thanks

Also in hdfs, the same . 

1 REPLY 1

Rising Star
* Do you have the e3base_kfapp in /etc/krb5.conf?
* You are using a short hostname for the zookeeper node, not a FQDN. This means it's likely going to use the default kerberos realm.
* Is there a domain mapping for the zookeeper address to the correct realm in /etc/krb5.conf