question Re: problem generating keytab with HTTP SPN in Support Questions

problem generating keytab with HTTP SPN

yagoaparecidoti — Mon, 22 Aug 2022 18:44:56 GMT

hello cloudera community,

we are trying to create a keytab with the main one:

"HTTP/hostname@DOMAIN.LOCAL"

with the command:

ktpass -princ HTTP/hostname@DOMAIN.LOCAL -mapuser livy-http -crypto ALL -ptype KRB5_NT_PRINCIPAL -pass password2022 -target domain.local -out c:\temp\livy-http.keytab

but I try to validate the ticket with this keytab returns the error:

Exception: krb_error 24 Pre-authentication information was invalid (24) Pre-authentication information was invalid

KrbException: Pre-authentication information was invalid (24)
at sun.security.krb5.KrbAsRep.<init>(Unknown Source)
at sun.security.krb5.KrbAsReqBuilder.send(Unknown Source)
at sun.security.krb5.KrbAsReqBuilder.action(Unknown Source)
at sun.security.krb5.internal.tools.Kinit.<init>(Unknown Source)
at sun.security.krb5.internal.tools.Kinit.main(Unknown Source)
Caused by: KrbException: Identifier doesn't match expected value (906)
at sun.security.krb5.internal.KDCRep.init(Unknown Source)
at sun.security.krb5.internal.ASRep.init(Unknown Source)
at sun.security.krb5.internal.ASRep.<init>(Unknown Source)
... 5 more

this user "livy-http" is already created in AD and with the SPN "HTTP/hostname@DOMAIN.LOCAL" attached to it

what are we doing wrong?

Re: problem generating keytab with HTTP SPN

JQUIROS — Mon, 22 Aug 2022 19:28:40 GMT

Hi sir,
This command is probably better to be evaluated in an AD forum, It is a power shell command in the AD server. Based on the stack trace you are getting, the pre-authentication is failing. Normally, this may happen because the account is enabled with pre-auth or you are using a cipher that requires pre-auth [0]

We can try to create by using only legacy ciphers:

##########################################
# How to Create a keytab from client application
##########################################

# Step 1: Type ktutil to enter prompt:
ktutil

# Step 2: At the ktutil prompt, add the authentication command below:
ktutil: add_entry -password -p livy-http@DOMAIN.LOCAL-k 1 -e arcfour-hmac-md5

# Step 3: Type password
Password for livy-http@DOMAIN.LOCAL:

# Step 4: Create Keytab file at ktutil prompt:
# ktutil: <command below to create keytab file>
wkt livy-http.keytab

# Step 5: Type quit to exit
quit

# Step 6: Verify Keytab Works Using kinit:
/usr/bin/kinit -V -kt livy-http.keytab livy-http@DOMAIN.LOCAL

[0] refer to the box checks "Do not required Kerberos Preauthentication": https://docs.informatica.com/data-integration/powercenter/10-2/security-guide/kerberos-authentication/preparing-to-enable-kerberos-authentication/step-3--create-kerberos-principal-accounts-in-active-directory.html

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Mon, 22 Aug 2022 19:36:31 GMT

hi@JQUIROS ,

should "kutil" command be run on cluster host or AD host?

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Mon, 22 Aug 2022 19:43:04 GMT

hi @JQUIROS

if create another keytab with the SPN below:

"livy-http/hostname@DOMAIN.LOCAL"

works, no problems.

the problem is when using HTTP

Re: problem generating keytab with HTTP SPN

JQUIROS — Mon, 22 Aug 2022 19:47:56 GMT

In regards to your first question, it is on the cluster host.

For your second, We only create the keytab against the service SPN ("livy-http/hostname@DOMAIN.LOCAL"), what is the business purpose to create the keytab with HTTP principals? The service is authenticating against Service Principals, not HTTP.

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Mon, 22 Aug 2022 19:51:06 GMT

hi @JQUIROS

we need to create the HTTP SPN keytab to use in the Livy service, as described in the link below:

https://enterprise-docs.anaconda.com/en/latest/admin/advanced/config-livy-server.html

in the link above, kadmin was used, but we don't have kadmin but AD.

Re: problem generating keytab with HTTP SPN

JQUIROS — Tue, 23 Aug 2022 00:09:22 GMT

ktpass might be purely AD, might be worth it to open an AD case if that is the only option.

Otherwise, Could you please try to create the keytab with the following ktutil commands:
add_entry -password -p HTTP@FQDN_DOMAIN.LO -k 1 -e arcfour-hmac-md5

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Tue, 23 Aug 2022 15:06:10 GMT

hi @JQUIROS

using the ktutil command it was possible to create the principal:

HTTP/hostname@DOMAIN.LOCAL

how to export keytab now?

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Tue, 23 Aug 2022 15:13:12 GMT

hi @JQUIROS

we were able to export the keytab with the command:

write_kt http.keytab

but when validating the ticket with the command:

kinit -kt http.keytab HTTP/hostnamae@DOMAIN.LOCAL

got the same error:

kinit: Preauthentication failed while getting initial credentials

Re: problem generating keytab with HTTP SPN

JQUIROS — Tue, 23 Aug 2022 15:58:35 GMT

Hi @yagoaparecidoti,
The error is coming directly from the Active Directory KDC, please limit the keytab to RC4 HMAC as commented earlier. Scroll up on the first post.
Then, try to kinit by using the trace to understand the issue better:

KRB5_TRACE=/dev/stdout kinit -kt http.keytab HTTP/hostnamae@DOMAIN.LOCAL

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Tue, 23 Aug 2022 16:21:16 GMT

hi @JQUIROS

the command to create the entry was:

add_entry -password -p HTTP/hostname@DOMAIN.LOCAL -k 1 -e rc4-hmac

then export the keytab with the command:

wkt http.keytab

and then to validate the tiker the command:

KRB5_TRACE=/dev/stdout kinit -kt http.keytab HTTP/hostname@DOMAIN.LOCAL

presented the error:

Getting initial credentials for HTTP/hostname@DOMAIN.LOCALLooked up etypes in keytab: rc4-hmac
Sending unauthenticated request
Sending request (237 bytes) to DOMAIN.LOCAL
Sending initial UDP request to dgram 172.22.22.22:88
Received answer (229 bytes) from dgram 172.22.22.22:88
Response was from master KDC
Received error from KDC: -1765328359/Additional pre-authentication required
Preauthenticating using KDC method data
Processing preauth types: PA-PK-AS-REQ (16), PA-PK-AS-REP_OLD (15), PA-ETYPE-INFO2 (19), PA-ENC-TIMESTAMP (2)
Selected etype info: etype rc4-hmac, salt "", params ""
Retrieving HTTP/hostname@DOMAIN.LOCAL from FILE:http.keytab (vno 0, enctype rc4-hmac) with result: 0/Success
AS key obtained for encrypted timestamp: rc4-hmac/20C1
Encrypted timestamp (for 1661441475.76781): plain 301AA011199992303232BED, encrypted 3625254347B405C2739999992C5C50F451C0A477AE3AD421DF
Preauth module encrypted_timestamp (2) (real) returned: 0/Success
Produced preauth for next request: PA-ENC-TIMESTAMP (2)
Sending request (313 bytes) to DOMAIN.LOCAL
Sending initial UDP request to dgram 172.22.22.22:88
Received answer (196 bytes) from dgram 172.22.22.22:88
Response was from master KDC
Received error from KDC: -1765328360/Preauthentication failed
Preauthenticating using KDC method data
Processing preauth types: PA-ETYPE-INFO2 (19)
Selected etype info: etype rc4-hmac, salt "", params ""
kinit: Preauthentication failed while getting initial credentials

Re: problem generating keytab with HTTP SPN

JQUIROS — Tue, 23 Aug 2022 16:55:30 GMT

We need to reach out the AD support, the response is coming from the AD

Response was from master KDC
Received error from KDC: -1765328360/Preauthentication failed
Preauthenticating using KDC method data
Processing preauth types: PA-ETYPE-INFO2 (19)
Selected etype info: etype rc4-hmac, salt "", params ""
kinit: Preauthentication failed while getting initial credentials

[0] Reference: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-kile/ae60c948-fda8-45c2-b1d1-a71b484dd1f7

Re: problem generating keytab with HTTP SPN

araujo — Fri, 26 Aug 2022 00:00:13 GMT

@yagoaparecidoti

What do you need the keytab with the HTTP principal for?

André

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Fri, 26 Aug 2022 12:29:12 GMT

hi @araujo

to use in the livy service, as requested in the processes in the links below:

https://danielfrg.com/blog/2018/08/spark-livy/
https://enterprise-docs.anaconda.com/en/latest/admin/advanced/config-livy-server.html

Re: problem generating keytab with HTTP SPN

araujo — Sat, 27 Aug 2022 01:55:45 GMT

@yagoaparecidoti ,

The problem is that to generate a keytab for any principal you need to know the password for that principal. The HTTP/hostname principal probably already exists in your AD and has some unknown password. Without knowing that you would have to reset the principal password to be able to create a keytab for it. And if you reset its password you will invalidate any keytabs that already exist for that principal that other services may be using.

Cheers,

André

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Mon, 29 Aug 2022 13:33:26 GMT

hi @araujo

the ad has two users:

livy

livy-http

the user livy has the SPN:

livy/hostname@DOMAIN.LOCAL

and it is working without problem in kinit

the user livy-http has the SPN:

HTTP/hostname@DOMAIN.LOCAL

but it is showing the error described above

Re: problem generating keytab with HTTP SPN

araujo — Mon, 29 Aug 2022 22:54:38 GMT

@yagoaparecidoti ,

Do you know the passwords for the users livy and livy-http? Can you manually kinit with those 2 users from the command line?

Can you also check in AD what's the value for userPrincipalName property of those two users and share it here?

Cheers,

André

Re: problem generating keytab with HTTP SPN

yagoaparecidoti — Tue, 30 Aug 2022 12:41:46 GMT

hi @araujo

yes, we know the passwords, because we created these two users from scratch

before creating the keytabs for the two users, we managed to kinit the two users without problem "kinit user", after creating the keytabs for the two users, kinit only works with the keytab, but it only works on the livy user, when we try to run kinit in livy-http user keytab displays the error "kinit: Preauthentication failed while getting initial credentials"

the userprincipalname of each user is:

livy:
livy/hostname_livy_server@DOMAIN.LOCAL

livy-http:
HTTP/hostname_livy_server@DOMAIN.LOCAL

Re: problem generating keytab with HTTP SPN

araujo — Tue, 30 Aug 2022 13:32:05 GMT

The names you listed are the servicePrincipalName. These are different from the userPrincipalName. Could you please check the latter and let me know what they are?

Cheers,

André

Re: problem generating keytab with HTTP SPN

araujo — Tue, 30 Aug 2022 13:36:26 GMT

Could you please run the kinit commands for both accounts and share a screenshot showing the command line and the output?