Would like some assistance/guidance on Kerberos. Our domain name has changed and since then our applications are unable to connect to Hadoop cluster. We are using MIT Kerberos.
Domain name changes will affect the KDC database. Kerberos is super sensitive to domain changes according to experience you will have to recreate the KDC database and regenerate the keytabs/principals to enable you applications to reconnect.
If the hosts in the cluster were re-named ie host1.old.com to host1.new.com then ensure those changes are also reflected or resolved by the DNS.
This is going a tricky one but fortunately, CM or Ambari will make your work easy now that your domain has changed the earlier generated keytabs have the old domain name . A keytab contains a pair of principals and an encrypted copy of that principal's key it's unique to each host since the principal names include the hostname and may be concatenated with the domain name
Delete the old KDC database
Usually, as the root user call the Kerberos database utility kdb5_util destroy assuming the old domain was OLD.COM this should delete the keytabs and principals linked to the old REALM,
# kdb5_util -r OLD.COM destroy
You will need to manually delete the keytabs liked to the old REALM on the file system /etc/security/keytabs/ [HDP] or /etc/hadoop/conf/[CDH]. You will be prompted to confirm before destroying the database, usually, this is a better option if you have second thought rather than using the kdb5_util destroy -f will naturally not prompt you for a confirmation
Recreate the New KDC database
Use the Kerberos database utility kdb5_util create [-s] assuming the new domain was NEW.COM
# kdb5_util NEW.COM create # kdb5_util -r NEW.COM create -s
With the -s option, kdb5_util will stash a copy of the master key in a stash file this allows a KDC to authenticate itself to the database utilities, such as kadmin, kadmind, krb5kdc, and kdb5_util best option.
Update Kerberos files.
Make sure you update the below files to reflect the new REALM assuming your MIT KDC server's domain isn't changed.
krb5.conf kdc.conf kadm5.acl Auth-to-local Rules jaas.conf files [if being used by applications]
Using CM or Ambari the process is straight forward.
Please let me know if you need more help