Why is kinit with a headless keytab failing?

[#1]

The commands being ran are below. Both fail.

[root@host1 ~]# sudo -u hdfs /usr/bin/kinit -k -t /etc/security/keytabs/hdfs.headless.keytab hdfs/host1.prod.myclient.com@CORP.DS.MYCLIENT.COM
kinit: Keytab contains no suitable keys for hdfs/host1.prod.myclient.com@CORP.DS.MYCLIENT.COM while getting initial credentials 

and

[user1@host2.prod /var/www/html]$ sudo -u hdfs /usr/bin/kinit -k -t /etc/security/keytabs/hdfs.headless.keytab
kinit: Client not found in Kerberos database while getting initial credentials

[#2 · Accepted solution]

You are using the wrong principal name for that keytab file. To see what keytab entries in that keytab file, use klist

klist -kte /etc/security/keytabs/hdfs.headless.keytab

This will list out the contents of the keytab file. There will be 1 or more lines indicating details about each keytab entry. The 2nd-to-last column will show the principal name for each entry. The last column will show the encryption algorithm used to create the keytab entry.

[#3]

Hi Terry,

in a secured cluster you have two types of keytabs or principals.

Headless and Service principals.

Headless principals are not bound to a specific host or node, they have the syntax: <service_name>-<clustername>@EXAMPLE.COM

Service princiapsl are bound to a specific service and host or node, they have the syntax: <service-name>/<hostname>@EXAMPLE.COM

For Example:

Headless: hdfs-mycluster@EXAMPLE.COM
Service: nn/c6601.ambari.apache.org@EXAMPLE.COM

Here is some more info https://docs.oracle.com/cd/E21455_01/common/tutorials/kerberos_principal.html

Make sure you use the right principal when you use kinit, you can see the principals of a keytab with

klist -k <keytab file>