question Re: Kerberos KDC not reachable in Support Questions

Kerberos KDC not reachable

arunpoy — Fri, 16 Sep 2022 10:10:40 GMT

kdc-unreachable.jpgI am trying to kereeberise my HDP cluster. I have installed a KDC on the ambari host itself and i want to use that. so i selected option 1 (existing KDC) in ambari. But when i try to test the KDC connection it fails and i get the following error.

23 Mar 2016 13:16:29,457 WARN [qtp-ambari-client-18131] KdcServerConnectionVerification:187 - An unexpected exception occurred while attempting to communicate with the KDC server at hostname:88 over TCP 23 Mar 2016 13:16:29,459 WARN [qtp-ambari-client-18131] KdcServerConnectionVerification:187 - An unexpected exception occurred while attempting to communicate with the KDC server at hostname:88 over UDP 23 Mar 2016 13:16:29,460 ERROR [qtp-ambari-client-18131] KdcServerConnectionVerification:113 - Failed to connect to the KDC at hostname:88 using either TCP or UDP

But when i try to do kinit or invoke any kerberos command from the ambari/KDC host it is working fine. It is pretty strange and i dont see any network related issues. this error is seen only when a wrong kdc information is provided. But in my case even after providing the correct details it fails. From the logs i couldnt trace anything.

Is there any way to debug or trace it.

i used ping, telenet to do the basic checks and everything is fine.

P.s i have just replaced my KDC host names with the string "hostname"in the above error message.

Re: Kerberos KDC not reachable

pcodding — Wed, 23 Mar 2016 21:35:22 GMT

Can you try restarting the ambari-server and retrying the kerberos wizard, but this time specifying the correct information the first time?

Re: Kerberos KDC not reachable

rlevas — Wed, 23 Mar 2016 22:39:21 GMT

Looking at the following error message, you specified the KDC host as "hostname" not the FQDN of the relevant host.

An unexpected exception occurred while attempting to communicate with the KDC server at hostname:88 over TCP

Rerun the Enable Kerberos Wizard and make sure you set the correct hostname for the KDC and admin hosts.

Re: Kerberos KDC not reachable

arunpoy — Thu, 24 Mar 2016 07:52:09 GMT

@Robert Levas, i have used the FQDN only.

Re: Kerberos KDC not reachable

arunpoy — Thu, 24 Mar 2016 18:31:25 GMT

@Paul Codding

, @Artem Ervits, @Robert Levas,

One strange thing i noticed is, i am able to connect to the same KDC from ambari running in a different environment. But ambari in that environment is 2.1 and the one from where i am not able to connect is 2.2.1

This is weird. How do i resolve this. IS there a way i can just uninstall ambari alone or how do i fix this. I dont think this is a bug in amabri 2.2.1. i have tried this in sandbox before and it worked fine.

Re: Kerberos KDC not reachable

arunpoy — Thu, 31 Mar 2016 13:21:59 GMT

IS this could be because of the ambari upgrade from 2.1 to 2.2?

Re: Kerberos KDC not reachable

rlevas — Tue, 26 Apr 2016 22:44:26 GMT

@ARUNKUMAR RAMASAMY

I think I lost track of this issue... sorry about that. Are you still having issue?

The version of Ambari shouldn't make a difference here. Yu should make sure that you can manually connect to the KDC from the command line of the host where Ambari is running. Maybe there is a DNS issue?

Make sure the /etc/krb5.conf file is set to point to your KDC, then issue some command like:

kadmin -p <ADMIN PRINCIPAL> -q "get_principal <ADMIN PRINCIPAL>"

For example:

# kadmin -p admin/admin@EXAMPLE.COM -q "get_principal admin/admin@EXAMPLE.COM"
Authenticating as principal admin/admin@EXAMPLE.COM with password.
Password for admin/admin@EXAMPLE.COM:
Principal: admin/admin@EXAMPLE.COM
Expiration date: [never]
Last password change: Mon Apr 25 16:11:27 UTC 2016
Password expiration date: [none]
Maximum ticket life: 1 day 00:00:00
Maximum renewable life: 0 days 00:00:00
Last modified: Mon Apr 25 16:11:27 UTC 2016 (root/admin@EXAMPLE.COM)
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 6
Key: vno 1, aes256-cts-hmac-sha1-96, no salt
Key: vno 1, aes128-cts-hmac-sha1-96, no salt
Key: vno 1, des3-cbc-sha1, no salt
Key: vno 1, arcfour-hmac, no salt
Key: vno 1, des-hmac-sha1, no salt
Key: vno 1, des-cbc-md5, no salt
MKey: vno 1
Attributes:
Policy: [none]

If it fails, you might get something like:

# kadmin -p admin/admin@EXAMPLE.COM -q "get_principal admin/admin@EXAMPLE.COM"
Authenticating as principal admin/admin@EXAMPLE.COM with password.
kadmin: Cannot contact any KDC for requested realm while initializing kadmin interface

Re: Kerberos KDC not reachable

davidlu1001 — Thu, 12 May 2016 12:41:02 GMT

I have the same problem...

Ambari Version: 2.2.2.0

HDP Version: 2.4.2.0-258

I can ping KDC from Ambari Server and telnet port 88/749 as well as running kadmin command without error:

/usr/bin/kadmin -s KDC_SERVER:749 -p admin -w ****** -r EXAMPLE.COM -q "get_principal admin"

I've also checked the src code but no luck:

https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/java/org/apache/ambari/server/KdcServerConnectionVerification.java

After retarting Ambari Server in DEBUG mode, found the problem:

Caused by: java.lang.IllegalArgumentException: Algorithm AES256 not enabled

Still testing how to enable it...

modify the krb.conf, but doesn't work.

It looks like the JCE installation issue (confirm the JCE policy jars were in the right place...)

Re: Kerberos KDC not reachable

arunpoy — Thu, 12 May 2016 14:19:28 GMT

Install JCE with unlimited strength.

http://www.oracle.com/technetwork/java/javase/downloads/index.html

I did this and the issue got resolved. after that i did a restart of ambari server.

Re: Kerberos KDC not reachable

alberto_garciag — Mon, 08 Aug 2016 14:12:44 GMT

It works great!

By the way, you can download the JCE in this url:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Security_Guide/content/_distribute_and_install_the_jce.html

Thanks!

Re: Kerberos KDC not reachable

hardikv_desai — Fri, 03 Mar 2017 16:14:55 GMT

It doesnt work for me even though i faced same problem and i did same steps mentioned..

Re: Kerberos KDC not reachable

alam_shakeen — Tue, 06 Apr 2021 05:51:58 GMT

Hi,

You must deploy the jce policy on every cluster nodes if you are using built-in openjdk.

Please follow the steps.

[root@hostname]# locate local_policy.jar

[root@hostname]# wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip"

[root@hostname]# unzip -o -j -q jce_policy-8.zip -d /usr/jdk64/jdk1.8.0_112/jre/lib/security/

[root@hostname]# ambari-server rstart

Now test the connection. Hopes it will resolves the issue.