.KerberosOperationException: Unexpected error condition executing the kadmin command. STDERR: kadmin: Matching credential not found

[#1]

Hello all,

I'm trying to kerberize the Ambari 2.7.3 cluster. However, during the setup, I get the following error:

Caused by: org.apache.ambari.server.serveraction.kerberos.KerberosOperationException: Unexpected error condition executing the kadmin command. STDERR: kadmin: Matching credential not found (filename: /tmp/ambari_krb_142308985016794830cc) while initializing kadmin interface
        at org.apache.ambari.server.serveraction.kerberos.MITKerberosOperationHandler.invokeKAdmin(MITKerberosOperationHandler.java:323)
        at org.apache.ambari.server.serveraction.kerberos.MITKerberosOperationHandler.principalExists(MITKerberosOperationHandler.java:123)
        at org.apache.ambari.server.serveraction.kerberos.KerberosOperationHandler.testAdministratorCredentials(KerberosOperationHandler.java:314)
        at org.apache.ambari.server.controller.KerberosHelperImpl.validateKDCCredentials(KerberosHelperImpl.java:2133)

All of the authentication settings are okay, because I am able to kinit and use the kadmin interface from shell.

It seems that the problem is that Ambari tries to do the following:

kinit -p admin/admin@EXAMPLE.COM
kadmin -c /tmp/ambari_krb_...

While it should be doing the following:

kinit -S kadmin/admin@EXAMPLE.COM admin/admin@EXAMPLE.COM
kadmin -c /tmp/ambari_krb...

I've tried replicating the two settings and confirmed my guess. The second code works from the shell. Further, If I intercept the temporarily generated credentials by ambari with my own, the code works.

How can I fix this behaviour? This seem like a bug in Ambari code -- which part should I edit to fix this?

[#2 · Accepted solution]

@Javert Kirilov In Ambari 2.7.x, the MIT KDC connector logic uses the following kinit format:

kinit -S kadmin/<FQDN kadmin server>@EXAMPLE.COM admin/admin@EXAMPLE.COM

See https://github.com/apache/ambari/blob/branch-2.7/ambari-server/src/main/java/org/apache/ambari/serve... for the code.

This is different then what you suggest since the server principal is basically hardcoded to kadmin/<FQDN kadmin server>@<REALM>. Since not all installations of the MIT KDC have this principal set up, this can cause issues like what you are seeing. In the newer release of Ambari, we have this fixed and the user can override the kadmin server principal. So until that version is release, it is suggested that you manually create that missing principal.

Hopefully you are willing to try this is see if it works for you.

[#3]

Thanks, I've noticed that too, after posting. While -S kadmin/admin worked, the -S kadmin/FQDN didn't. So reconfiguring this part on the KDC solved the problem. It's just interesting that I didn't bump into this on HDP 2.6 Ambari.

About the future release of Ambari -- any ETA yet? 🙂

[#4]

I do not have any information on in the release date of the next version of Ambari.

This way of authenticating to the kadmin server is new for Ambari 2.7, so your issue would not have been seen in previous versions

[#5]

@mRabramS Can you share how you reconfigure the KDC ? 

@VidyaSargur I’m really sorry , I opened a new thread but there’s no response 

BR,

[#6]

Still have the same issue , 

I’m sure the configuration files are okay!

BR,

[#7]

@Aziz_G as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.