question Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file in Support Questions

AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

shyamshaw — Fri, 16 Sep 2022 11:46:21 GMT

We have configured Hive LLAP in our existing environment for testing purpose. Cluster setup has been done using Service User Accounts present in AD. We tried to Enable Kerberos today and it got failed in Create Keytabs steps with below error message:

stderr: errors-257.txt

2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

stdout: output-257.txt

2017-06-15 20:09:24,315 - Processing identities...
2017-06-15 20:09:24,378 - Creating keytab file for hive_llap-061517@TESTING.COM on host master1.TESTING.com
2017-06-15 20:09:24,380 - Creating keytab file for hive_llap-061517@TESTING.COM on host data1.TESTING.com

It seems that ambari is trying to create keytabs on linux hosts. we configured hive LLAP through ambari only.

Any help on this issue?

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

rlevas — Fri, 16 Jun 2017 04:12:50 GMT

It appears that there may be an _accounting_ issue in the Ambari database. This may have been created due to a previous failure when enabling Kerberos.

Try to remove the entries in the kerberos_principal_host and kerberos_principal where the principal_name column is hive_llap-061517@TESTING.COM in the Ambari database. Then restart Ambari and try to enabled Kerberos again.

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

dsun — Fri, 16 Jun 2017 04:23:15 GMT

Assuming all the other services are working properly except for Hive LLAP. You can try to 'Regenerate Keytabs' in Ambari. While doing that, you need to check 'Only regenerate keytabs for missing hosts and components' option.

If that doesn't work, you can regenerate all the keytabs for all hosts which requires all components to be restarted.

Also, if you are using a test AD KDC, you can try to restart it by following these instructions. Hope that helps.

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

shyamshaw — Fri, 16 Jun 2017 20:35:12 GMT

@Robert Levas

Thanks for the quick response. After removing entries in the kerberos_principal_host and kerberos_principal, it worked.

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

rlevas — Fri, 16 Jun 2017 21:06:16 GMT

@Shyam Shaw Awesome. I am glad it worked.

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

rohan_goel — Wed, 28 Nov 2018 22:59:58 GMT

Hi @Robert Levas @Shyam Shaw, can you please help me with how to "remove the entries in the kerberos_principal_host and kerberos_principal where the principal_name column". Sorry for re-opening this, but I am facing a similar problem and unable to delete this from ambari database.

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

rlevas — Wed, 28 Nov 2018 23:13:39 GMT

@Rohan Goel

If you are using Ambari before version 2.7.0, then you can do the following:

delete from kerberos_principal_host where principal_name = 'THE PRINCIPAL NAME";
delete from kerberos_principal where principal_name = 'THE PRINCIPAL NAME";

If you are using Ambari, version 2.7.0 and above, then you have to do the following:

delete from kerberos_keytab_principal where principal_name = 'THE PRINCIPAL NAME";
delete from kerberos_principal where principal_name = 'THE PRINCIPAL NAME";

Then restart Ambari.

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

sandrine — Thu, 15 Oct 2020 08:58:58 GMT

Hello,

I'm rencontring the same issue , I tried to remove keberos entries,but I still have the same issue. Any help please?

2020-10-15T10:42:37.386+0200, User(admin), Operation(CUSTOM_COMMAND KERBEROS_CLIENT), Details(CHECK_KEYTABS KERBEROS/KERBEROS_CLIENT), Status(COMPLETED), RequestId(685), TaskId(7814), Hostname(ppla0016.angers.cnp.fr)
2020-10-15T10:42:37.421+0200, User(admin), Operation(CUSTOM_COMMAND KERBEROS_CLIENT), Details(CHECK_KEYTABS KERBEROS/KERBEROS_CLIENT), Status(COMPLETED), RequestId(685), TaskId(7810), Hostname(ppla0010.angers.cnp.fr)
2020-10-15T10:42:37.424+0200, User(admin), Operation(CUSTOM_COMMAND KERBEROS_CLIENT), Details(CHECK_KEYTABS KERBEROS/KERBEROS_CLIENT), Status(COMPLETED), RequestId(685), TaskId(7812), Hostname(ppla0012.angers.cnp.fr)
2020-10-15T10:42:37.955+0200, User(admin), Operation(EXECUTE AMBARI_SERVER_ACTION), Details(Create Keytabs), Status(QUEUED), RequestId(685), TaskId(7817), Hostname(null)
2020-10-15T10:42:37.964+0200, User(admin), Operation(EXECUTE AMBARI_SERVER_ACTION), Details(Create Keytabs), Status(IN_PROGRESS), RequestId(685), TaskId(7817), Hostname(null)
2020-10-15T10:42:39.552+0200, Operation(Keytab file creation), Status(Failed), Reason of failure(Failed to create keytab for clhdpi01-101520@P.AD.CNP.FR, missing cached file), RequestId(685), TaskId(7817), Principal(clhdpi01-101520@P.AD.CNP.FR), Hostname(ppla0017.angers.cnp.fr), Keytab file(/etc/security/keytabs/kerberos.service_check.101520.keytab)
2020-10-15T10:42:39.557+0200, User(admin), Operation(Kerberos Service Check), Status(FAILED), RequestId(685)
2020-10-15T10:42:39.557+0200, User(admin), Operation(EXECUTE AMBARI_SERVER_ACTION), Details(Create Keytabs), Status(FAILED), RequestId(685), TaskId(7817), Hostname(null)
2020-10-15T10:51:20.156+0200, User(admin), RemoteIp(10.68.123.240), Operation(Configuration change), RequestType(PUT), url(XXXXXXXX), ResultStatus(200 OK), VersionNumber(Vnull), VersionNote(null)
2020-10-15T10:51:20.519+0200, User(admin), RemoteIp(10.68.123.240), Operation(Service deletion), RequestType(DELETE), url(XXXXX), ResultStatus(200 OK), Service(KERBEROS)

Re: AD Kerberos setup is failing in Create Keytabs step. Error Message: 2017-06-15 20:09:24,381 - Failed to create keytab for hive_llap-061517@TESTING.COM, missing cached file

vijaykumar — Thu, 08 Jul 2021 10:38:01 GMT

1. Login to ambari database hosted server.

2. Take the backup of database. Replace XXXXXX with correct pasword

nohup mysqldump -u root -pXXXXXX --databases ambari >/ambari.sql &

3. Login to mysql with root or ambari account and remove hive keytabs.

delete from kerberos_principal_host where principal_name like '%hive%';

delete from kerberos_principal where principal_name like '%hive%';

4. Restart Ambari server.

5. Regenerate the keytabs with valid account

6. Start the Node manager.

Note: its not only for Hive.. we can remove based on error. as caches in ambari database prevents to regenerate again