KDC Default Realm in krb5.conf

[#1]

We have several realms in our company and we plan to dedicate one to our Hadoop cluster managed by Ambari. Let's say we have:

PROD.COM #default realm to be used by production services
HADOOP.COM #dedicated for Hadoop cluster

And it is mandatory for us to have PROD.COM as default realm in krb5.conf. However with PROD.COM as default realm I always get this error after successful kinit as hdfs:

[root@hadoop-poc2-01:/etc] kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-poc2@HADOOP.COM

[root@hadoop-poc2-01:/etc] hadoop fs -ls /
16/07/11 13:28:39 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
16/07/11 13:28:40 WARN ipc.Client: Couldn't setup connection for hdfs-poc2@HADOOP.COM to hadoop-poc2-01.int.na.prod.com/172.30.52.136:8020
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds)]

With default realm set to HADOOP.COM it just works.

Any hints? Thx

[#2 · Accepted solution]

@Milan Sladky

If you add a [domain_realm] section to your krb5.conf file, mapping the domain or DNS names of the hosts in the cluster to the HADOOP.COM realm, this should help:

Example using just the domain portion of the FQDNs of the hosts in the hadoop cluster

[domain_realm]
.my.hadoop.domain.com = HADOOP.COM

Example using the explicit FQDNs of the hosts in the hadoop cluster

[domain_realm]
hadoop-poc2-01.my.hadoop.domain.com = HADOOP.COM
hadoop-poc2-02.my.hadoop.domain.com = HADOOP.COM

[#3]

@Robert Levas, thanks for hint. This actually works!

But I was afraid that following kinit (used a lot internally):

[root@hadoop-poc2-01:/etc] kinit host/hadoop-poc2-01.my.hadoop.domain.com

will be towards HADOOP.COM realm; based on the domain_realm settings...

But it actually goes towards PROD.COM realm:

[root@hadoop-poc2-01:/etc] kinit host/hadoop-poc2-01.my.hadoop.domain.com
Password for host/hadoop-poc2-01.my.hadoop.domain.com@PROD.COM: 

Which is good, but I do not understand why it works....

[#4]

Hi @Milan Sladky... It is unclear whether this solves your problem or not.

So when

kinit host/hadoop-poc2-01.my.hadoop.domain.com

is invoked after my domain_realm suggestion is apply, the assumed realm should be PROD.COM, not HADOOP.COM. This is because no explicit realm or domain is indicated in the principal name... the "/hadoop-poc2-01.my.hadoop.domain.com" part of the principal name does not get used by the underlying libraries to determine the relevant host, domain, or realm. Therefore, the default realm value is used.

[#5]

@Robert Levas, that explains it just fine. Thx for all help, really appreciated.