Ambari enable Kerberos does not create principals

[#1]

Hi,

I am using Ambari 2.0.1 and MIT Kerberos.

After running through the enabling Kerberos wizard, the services are failing to start. After some search I found out that there are no principals being created in the KDC:

"listprincs" just shows the previously (manually) created admin/admin@REALM principal, but no further principals as expected from enabling Kerberos via the wizard?!?!?!

This is the first time I see this strange behaviour, several other kerberized clusters didn't have this problem.

Why doesn't the Ambari wizard create principals in the KDC, while showing no errors at running through the wizard ?

Thanks in advance...

[#2]

@Gerd Koenig anything in the Ambari Server log files you see? I'd suggest if you do not have any issues moving to Ambari 2.1.2.1 - it would be worth a try with that.

[#3]

Hello @mahadev ,

no, the logs aren't that verbose, I just see:

12 Dec 2015 20:38:51,803 ERROR [ambari-action-scheduler] ClusterImpl:2382 - ServiceComponentHost lookup exception
12 Dec 2015 20:38:51,810  INFO [Server Action Executor Worker 1179] KerberosServerAction:327 - Processing identities...
12 Dec 2015 20:38:52,032  INFO [Server Action Executor Worker 1179] KerberosServerAction:429 - Processing identities completed.
12 Dec 2015 20:38:52,839 ERROR [ambari-action-scheduler] ClusterImpl:2382 - ServiceComponentHost lookup exception
12 Dec 2015 20:38:52,847  INFO [Server Action Executor Worker 1180] KerberosServerAction:327 - Processing identities...
12 Dec 2015 20:38:52,848  INFO [Server Action Executor Worker 1180] CreateKeytabFilesServerAction:170 - Creating keytab file for HTTP/deala01875.domain@HDP.SIT on host deala01875.comain

And afterwards the keytab files are being created...

I ran a "disable Kerberos" afterwards, and then in the ambari-logs I can see that it tries to delete all the principals, but somehow the creation fails. I tried the whole steps several times....

Regards..

[#4]

Is Ambari running as non-root? If so, make sure the user that Ambari runs as, has authorization to write to /var/lib/ambari-server/tmp.

Also, did you change the value of the encryption types field? I think that it is under Advanced Kerberos-env section when on the Configure Kerberos page of the wizard. If this field is blank, then no encryption types will be specified and thus no keytab entries for the keytab files will be generated. If you edited the field, try setting it back to aes des3-cbc-sha1 rc4 des-cbc-md5 when you enable Kerberos again.

[#5]

Hi Robert, thanks for answering.

Ambari was running as root. "was" because I did a reinstall from scratch in the meantime due to time pressure for delivering the cluster....unfortunately.

This new installation worked nice, therefore I guess the problem was caused by "disable Kerberos" and maybe that was done in a corrupt manner...?!?!

[#6]

Hello, I am again facing the same issue while enabling Kerberos on a newly installed cluster => no principals are being created and no keytabs are generated, although the enable Kerberos wizard tells so ?!?!?

I didn't edit the encryption type field in the Kerberos-wizard and Ambari is running as root, therefore it will be able to write to /var/lib/ambari-server/tmp

Ambari logfile states the creation of the keytab files:

25 Jan 2016 12:01:48,279  INFO [Server Action Executor Worker 2148] CreateKeytabFilesServerAction:170 - Creating keytab file for HTTP/b0d05g22.<domain>@<realm> on host b0d05g22.<domain>
25 Jan 2016 12:01:48,280  INFO [Server Action Executor Worker 2148] CreateKeytabFilesServerAction:170 - Creating keytab file for hdfs@<realm> on host b0d05g22.<domain>

but at the end, no keytab file is being deployed, and also no principal has been created.

If I check the principals AFTER the Kerberos-Wizard has "successfully" created them, none of the are in the KDC =>

sudo kadmin.localkadmin.local:  listprincs
K/M@<realm>
admin/admin@<realm>
kadmin/admin@<realm>
kadmin/b0d095j2.<domain>@<realm>
kadmin/changepw@<realm>
krbtgt/HDP.ZURICH.PRD@<realm>

kadmin.local:

Kerberos client conf contains:

[libdefaults]
  renew_lifetime = 7d
  forwardable = true
  default_realm = <realm>
  ticket_lifetime = 24h
  dns_lookup_realm = false
  dns_lookup_kdc = false
  #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
  #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5

What to check further...Any hint highly appreciated...

[#7]

What version of Ambari are you using?

[#8]

@Robert Levas : it is Ambari2.0.1 (in combination with HDP2.2.4.2)

[#9]

@Gerd Koenig

Try this...

1. Disable Kerberos using the Ambari UI
2. Remove the stored Kerberos Descriptor via the Ambari API
3. Re-enable Kerberos using the Ambari UI

To remove the Kerberos Descriptor:

curl -u admin:admin -i -H 'X-Requested-By: ambari' -X DELETE http://AMBARI_SERVER:8080/api/v1/clusters/CLUSTER_NAME/artifacts/kerberos_descriptor

• Change the admin credentials (-u admin:admin) to match your administrative Ambari account
• Change the host and port (AMBARI_SERVER:8080) to point to your Ambari instance
• Change the cluster name (CLUSTER) to the relevant cluster name

[#10]

Hi @Robert Levas , thanks for this hint.

I did exactly that, but ended up in the same situation.

No principals have been created and no keytabs have been deployed, although the wizard marked every step as "green" until starting up the services