@harish
Yes for sure, that's doable I am assuming you have set up 2 kdc's on different networks but accessible to the cluster,
Assumptions:
You MUST have successfully configure the 2 master and slave KDC's
my realm =REALM
Master host=master-kdc.test.com
Slave host=slave-kdc.test.com
Contents of /var/kerberos/krb5kdc/kpropd.acl:
host/master-kdc.test.com@REALM
host/slave-kdc.test.com@REALM
# Create the configuration for kpropd on both the Master and Slave KDC hosts:
# Create /etc/xinetd.d/krb5_prop with the following contents.
service krb_prop
{
disable = no
socket_type = stream
protocol = tcp
user = root
wait = no
server = /usr/sbin/kpropd
} # Configure xinetd to run as a persistent service on both the Master and Slave KDC hosts:
# systemctl enable xinetd.service
# systemctl start xinetd.service
# Copy the following files from the Master KDC host to the Slave KDC host:
/etc/krb5.conf
/var/kerberos/krb5kdc/kadm5.acl
/var/kerberos/krb5kdc/kdc.conf
/var/kerberos/krb5kdc/kpropd.acl
/var/kerberos/krb5kdc/.k5.REALM
# Perform the initial KDC database propagation to the Slave KDC:
# kdb5_util dump /usr/local/var/krb5kdc/slave_datatrans
# kprop -f /usr/local/var/krb5kdc/slave_datatrans slave-kdc.REALM
# Start the Slave KDC :
# systemctl enable krb5kdc
# systemctl start krb5kdc
# Script to propagate the updates from the Master KDC to the Slave KDC. Create a cron job, or the like, to run this script on a frequent basis.
#!/bin/sh
#/var/kerberos/kdc-slave-propogate.sh
kdclist = "slave-kdc.customer.com"
/sbin/kdb5_util dump /usr/local/var/krb5kdc/slave_datatrans
for kdc in $kdclist
do
/sbin/kprop -f /usr/local/var/krb5kdc/slave_datatrans $kdc
done
How to test the KDC HA is to shut down the master KDC as start the slave KDC note both KDC's should NEVER be running at the same time, the crontab script should do the propagation of all changes in the KDC database in the master to the slave.
CAUTION
Run the kprop before shutting down the master KDC then to test the kdc HA log on to the cluster linux CLI follow the below steps my steps I am using the root user
Switch user to hive/spark/Yarn etc
# su - hive
Check if the hive user still has valid Kerberos ticket The below output shows the hive user still has a valid ticket
$ klist
Ticket cache: FILE:/tmp/krb5cc_507
Default principal: hdfs-host1@{REALM}
Valid starting Expires Service principal
12/28/16 22:57:11 12/29/16 22:57:11 krbtgt/{REALM}@{REALM} renew until 12/28/16 22:57:11
12/28/16 22:57:11 12/29/16 22:57:11 HTTP/host1.test.com@{REALM} renew until 12/28/16 22:57:11
12/28/16 22:57:11 12/29/16 22:57:11 HTTP/host1.com@{REALM} renew until 12/28/16 22:57:11 # Destroy the Kerberos tickets as user hive
$ kdestroy
Running the previous command shouldn't give you any lines, now try getting a valid ticket by running the following command format {kinit -kt $keytab $principal}
$ kinit -kt /etc/security/keytabs/hive.keytab {PRINCIPAL} Repeating the klist should give the hive user a valid ticket this will validate that the HA is functioning well.
