Troubleshooting one-way trust from KDC to AD

[#1]

Is there any way to troubleshoot & find out what's wrong with one-way trust from a KDC to AD? My problem is that the AD domain is set in lower-case letters: pqr-net.com. KDC on the cluster side is up and running and the cluster is kerberized against the KDC and works fine. Users registered on KDC can use the cluster without problems. For AD users, I followed the steps from documentation and from here. My HDP realm is HDP-NET.COM. As an additional realm in my kdc5.conf I have set: PQR-NET.COM in capitals and I can do "kinit aduser1@PQR-NET.COM" and obtain a ticket. [I also tried to set a domain in lower-case letters like pqr-net.com but in that case kinit doesn't work.] So, aduser1 can get a ticket, but cannot access the cluster: "hdfs dfs -ls" returns:

17/02/14 13:02:37 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
17/02/14 13:02:37 WARN ipc.Client: Couldn't setup connection for aduser1@PQR-NET.COM to h1002.pqr-net.com/192.168.31.167:8020
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds)]

where h1002.pqr-net.com/192.168.31.167:8020 is my active NN. On AD I did:

ksetup /addkdc HDP-NET.COM kdchost.pqr-net.com
netdom trust HDP-NET.COM /Domain:PQR-NET.COM /add /realm /passwordt:mypassword

and in my KDC I created a principal "krbtgt/HDP-NET.COM@PQR-NET.COM" and set his password to "mypassword". I have also added rules in my auth_to_local for AD users. Besides the error above, the only other error I could find was in krb5kdc.log, but only for a short period of time, it doesn't appear any more:(Error): TGS_REQ: UNKNOWN SERVER: server='krbtgt/HDP-NET.COM@PQR-NET.COM'. I suspect the problem is the AD domain in lower-case letters, but I'm not sure. Any help will be appreciated.

[#2]

Debug log says that I have "error Message is KDC has no support for encryption type" on krbtgt/HDP-NET.COM@PQR-NET.COM and the type given as "EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType". But I have that set in my krb5.conf as an enctype, and AD supports it as well. No idea...

[#3]

Hello @Predrag Minovic,

Maybe an encryption problem with the ticket between the kdc and AD, try creating it with the following command

addprinc -e "aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal" krbtgt/HDP-NET.COM@PQR-NET.COM

Also check your /etc/krb5.conf and check that you have an entry for "PQR-NET.COM" and is correctly configured.

[#4]

Hi @Juan Manuel Nieto, thanks for your reply. I tried that, and several other values for "-e", but the error is the same, like below. Btw. when I do "klist -e user1@PQR-NET.COM" it says that encryption is aes256-cts-hmac-sha1-96. getprinc on krbtgt/LOCAL@PQR-NET.COM also returns both aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 and some other types. AD runs on Win-2008 and is supposed to support these types.

Btw, "default_tgs_enctypes: 18 17 16 23" I found here. 18 stands for aes256-cts-hmac-sha1-96. In the log below, 192.168.120.120 is my AD server.

>>> Credentials acquireServiceCreds: main loop: [0] tempService=krbtgt/HDP-NET.COM@PQR-NET.COM
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbKdcReq send: kdc=192.168.120.120 UDP:88, timeout=30000, number of retries =3, #bytes=1411
>>> KDCCommunication: kdc=192.168.120.120 UDP:88, timeout=30000,Attempt =1, #bytes=1411
>>> KrbKdcReq send: #bytes read=97
>>> KdcAccessibility: remove 192.168.120.120
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
         sTime is Wed Feb 15 16:38:11 JST 2017 1487144291000
         suSec is 949340
         error code is 14
         error Message is KDC has no support for encryption type
         sname is krbtgt/HDP-NET.COM@PQR-NET.COM
         msgType is 30
>>> Credentials acquireServiceCreds: no tgt; searching thru capath
>>> Credentials acquireServiceCreds: inner loop: [1] tempService=krbtgt/LOCAL@PQR-NET.COM
...

[#5]

@Predrag Minovic

I have some questions.

• Can you post your /var/kerberos/krb5kdc/kdc.conf?
• Have you deployed unlimited security policy JCE in every node?

[#6]

@Juan Manuel Nieto, Keep mind that the AD domain (aka realm name) is all lowercase characters. By convention, the name should be all uppercase characters. I believe that this is causing the issue since (I think) the underling MIT Kerberos libraries (krb5-libs 1.10.3) assume the uppercase naming convention.

Have you successfully integrated with an AD (Windows 2012) that uses a lowercase domain name?

[#7]

Is this one way trust encrypted between kdc and ad?

[#8]

This worked after checking "The other domain supports Kerberos AES Encryption" check-box on the trusted domain property dialog on AD. So, doing just "ksetup /setenctypeattr AES..." is not enough (this appears only to update a cell in Windows registry). Details here.