Kerberos High Availability Functionality Testing

[#1]

Is there a way to test MIT kerberos high availability functionality?Any approaches?

thanks in advance.

[#2]

Thank for your information @Geoffrey Shelton Okot

[#3]

Hi @Geoffrey Shelton Okot,

While testing Kerberos High Availability ,i stopped Master KDC server..and started kadmin on slave, and created a new principal (on slave).when i tried to get kerberos ticket for new principal ,it is failing with below error.

Error: kinit: Client 'user@RXSKNY.HDP.XX.COM' not found in Kerberos database while getting initial credentials

Any thoughts on this?

Thanks.

[#4]

@harish

Was the creation successful without errors? If so can you list the principals in the slave KDC

On the slave

# kadmin.local
kadmin:  listprincs

Is your slave KDC also in the krb5.conf?

In the format.

[realms]
  EXAMPLE.COM = {
    ...
    kdc = kdc1.example.com
    kdc = kdc2.example.com
    ...
  }

Because multiple KDCs may exist for an installation (failover, high availability, etc, Ambari should allow a user to specify multiple KDC hosts to be set while enabling Kerberos and updating the Kerberos service's configuration.

This should be done by allowing kerberos-env/kdc_host to accept a (comma-)delimited list of hosts and then parsing that list properly when building the krb5.conf file where each kdc_host item generates an entry in the relevant realm block

kerberos-env

And in Ambari your kerberos.env should have an entry the

{
  ...
 "kdc_hosts" : "kdc1.example.com, kdc2.example.com"
  ...
}

Some Ambari KDC documentation

To backup a KDC database to a file, use krb5_util_dump.

# kdb5_util dump kdcfile 

To restore the KDC database from the dump file created in the above step, do the following:

# kdb5_util load kdcfile

Please revert

[#5]

Hi @Geoffrey Shelton Okot

The new principal creation was successful and i can listprincs it with no issues.After enabling kerberos HA ,i had added secondary host in ambari,now every krb5.conf file is updated with two hosts like above.But the the new principal created on slave after disabling master KDC is failing with no credentials found error.

thanks.

[#6]

Hi @Geoffrey Shelton Okot

The principal creation was successful and i added secondary host in ambari after enabling Kerberos HA,every krb5.conf file in the cluster is modified with two KDC hosts as you mentioned above.But the new principal created on slave KDC(after disabling master) is not working as expected.

Thanks.

[#7]

@harish

Can you share with me exactly how you created the new principal and keytab? I would like to see the syntax remember to garble sensitive info.

Also remember to add a new entry in the cron kprop to also propagate the newly create principal in the slave KDC database to the Primary so in case you switch back everything is in sync !!

[#8]

Hey @Shelton, 

Thank you so much this article helps me a lot to configure KDC HA in Cloudera  6.1.1 CDH. The only one issue I am facing right now is when I stop my Master KDC and create a new principal using slave KDC it works fine but when I UP Master KDC the new created principal is deleted because it back to its previous state can you please share the script or cron tab configuration for syncing both KDC if one down than other will syn newly created principal when ever its up.

[#9]

@USMAN_HAIDER 

When you create a new Principal in the slave KDC you should also have a  crontab  that will propagate it to the master

#!/bin/sh 
#/var/kerberos/kdc-master-propogate.sh 
kdclist = "slave-kdc.customer.com" 
  /sbin/kdb5_util dump /usr/local/var/krb5kdc/master_datatrans 
  for kdc in $kdclist 
  do 
 /sbin/kprop -f /usr/local/var/krb5kdc/master_datatrans $kdc 
done 

This way the  principals will be sync'ed