question HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96) in Support Questions

HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)

rizalt — Thu, 18 Jul 2024 04:17:03 GMT

Hey everyone, after enabling Kerberos resource manager can't run, this log after try run resource manager. please Advice

File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 295, in _run_command raise WebHDFSCallException(err_msg, result_dict) resource_management.libraries.providers.hdfs_resource.WebHDFSCallException: Execution of 'curl -sS -L -w '%{http_code}' -X GET -d '' -H 'Content-Length: 0' --negotiate -u : 'http://master.hadoop.com:50070/webhdfs/v1/services/sync/yarn-ats?op=GETFILESTATUS'' returned status_code=403. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)</title> </head> <body><h2>HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)</h2> <table> <tr><th>URI:</th><td>/webhdfs/v1/services/sync/yarn-ats</td></tr> <tr><th>STATUS:</th><td>403</td></tr> <tr><th>MESSAGE:</th><td>GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)</td></tr> <tr><th>SERVLET:</th><td>com.sun.jersey.spi.container.servlet.ServletContainer-6f19ac19</td></tr> </table> </body> </html>

for additional informations
/etc/krb5.conf

[libdefaults] # renew_lifetime = 7d forwardable = true default_realm = EXAMPLE.COM ticket_lifetime = 24h dns_lookup_realm = false dns_lookup_kdc = false default_ccache_name = /tmp/krb5cc_%{uid} # default_tgs_enctypes = aes256-cts # default_tkt_enctypes = aes256-cts #permitted_enctypes = aes256-cts #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 permitted_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 [domain_realm] example.com = EXAMPLE.COM [logging] default = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log kdc = FILE:/var/log/krb5kdc.log [realms] EXAMPLE.COM = { master_kdc = master1.hadoop.com admin_server = master1.hadoop.com kdc = master1.hadoop.com }

Re: HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)

shubham_sharma — Fri, 19 Jul 2024 14:35:40 GMT

You can check if the keytabs created for resource manager is equipped with AES256 encryption type or not.

Check your keytabs using below command after taking the kerberos ticket using kinit-

klist -e

Re: HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)

rizalt — Mon, 22 Jul 2024 00:40:31 GMT

Thanks @shubham_sharma for the reply, I checked keytabs please see below

root@master:~# kinit rm/master.hadoop.com Password for rm/master.hadoop.com@EXAMPLE.COM: root@master:~# klist -e Ticket cache: FILE:/tmp/krb5cc_0 Default principal: rm/master.hadoop.com@EXAMPLE.COM Valid starting Expires Service principal 07/22/2024 00:32:44 07/22/2024 10:32:44 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 07/23/2024 00:32:40, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

still the error, please advices

Re: HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)

shubham_sharma — Mon, 22 Jul 2024 19:46:42 GMT

Hi @rizalt

There can me mismatch between your AD account and krb5.conf for encryption types[1]. Kindly check with your AD admin.

[1] https://learn.microsoft.com/en-us/archive/blogs/openspecification/windows-configurations-for-kerberos-supported-encryption-type

Re: HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)

rizalt — Tue, 23 Jul 2024 01:03:02 GMT

Thanks for the reply @shubham_sharma, I'm not using AD account just kerberos