Unexpected error condition executing the kadmin command. STDERR: kadmin: Matching credential not found

[#1]

Hello all,

I'm trying to kerberize the Ambari 2.7.3 cluster. However, during the setup, I get the following error:

Caused by: org.apache.ambari.server.serveraction.kerberos.KerberosOperationException: Unexpected error condition executing the kadmin command. STDERR: kadmin: Matching credential not found (filename: /tmp/ambari_krb_142308985016794830cc) while initializing kadmin interface
        at org.apache.ambari.server.serveraction.kerberos.MITKerberosOperationHandler.invokeKAdmin(MITKerberosOperationHandler.java:323)
        at org.apache.ambari.server.serveraction.kerberos.MITKerberosOperationHandler.principalExists(MITKerberosOperationHandler.java:123)
        at org.apache.ambari.server.serveraction.kerberos.KerberosOperationHandler.testAdministratorCredentials(KerberosOperationHandler.java:314)
        at org.apache.ambari.server.controller.KerberosHelperImpl.validateKDCCredentials(KerberosHelperImpl.java:2133)

All of the authentication settings are okay, because I am able to kinit and use the kadmin interface from shell.

It seems that the problem is that Ambari tries to do the following:

kinit -p admin/admin@EXAMPLE.COM
kadmin -c /tmp/ambari_krb_...

While it should be doing the following:

kinit -S kadmin/admin@EXAMPLE.COM admin/admin@EXAMPLE.COM
kadmin -c /tmp/ambari_krb...

I've tried replicating the two settings and confirmed my guess. The second code works from the shell. Further, If I intercept the temporarily generated credentials by ambari with my own, the code works.

How can I fix this behaviour? This seem like a bug in Ambari code -- which part should I edit to fix this?

[#2 · Accepted solution]

@Javert Kirilov In Ambari 2.7.x, the MIT KDC connector logic uses the following kinit format:

kinit -S kadmin/<FQDN kadmin server>@EXAMPLE.COM admin/admin@EXAMPLE.COM

See https://github.com/apache/ambari/blob/branch-2.7/ambari-server/src/main/java/org/apache/ambari/serve... for the code.

This is different then what you suggest since the server principal is basically hardcoded to kadmin/<FQDN kadmin server>@<REALM>. Since not all installations of the MIT KDC have this principal set up, this can cause issues like what you are seeing. In the newer release of Ambari, we have this fixed and the user can override the kadmin server principal. So until that version is release, it is suggested that you manually create that missing principal.

Hopefully you are willing to try this is see if it works for you.

[#3 · Accepted solution]

@Javert Kirilov In Ambari 2.7.x, the MIT KDC connector logic uses the following kinit format:

kinit -S kadmin/<FQDN kadmin server>@EXAMPLE.COM admin/admin@EXAMPLE.COM

See https://github.com/apache/ambari/blob/branch-2.7/ambari-server/src/main/java/org/apache/ambari/serve... for the code.

This is different then what you suggest since the server principal is basically hardcoded to kadmin/<FQDN kadmin server>@<REALM>. Since not all installations of the MIT KDC have this principal set up, this can cause issues like what you are seeing. In the newer release of Ambari, we have this fixed and the user can override the kadmin server principal. So until that version is release, it is suggested that you manually create that missing principal.

Hopefully you are willing to try this is see if it works for you.

[#4 · Accepted solution]

Hello!

ran into the same problem, but in my case the kadmin/<FQDN kadmin server>@<REALM> existed and if i run

kinit -S kadmin/@<REALM> admin/admin@<REALM>

then he will authorize

# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin/admin@<REALM>

Valid starting       Expires              Service principal
08/07/2019 11:30:25  08/07/2019 14:30:25  kadmin/cluster@<REALM>