Created 11-08-2017 05:01 PM
It would be very interesting can you share the link you used to setup the KDC? Do the 3 KDC's have the same REALM is so how do they synchronise the KDC database?
How did you set up the trust between the 2 clusters? Did you follow the same procedure like below
Setting up a Realm
Trust I am wondering how you set up the trust between the 2 cluster? In my example here, the Kerberos realms are A.EXAMPLE.COM and B.EXAMPLE.COM.
You need to create the entry for the shared principal for the B realm in the A realm, using kadmin.
# kadmin -r A.EXAMPLE.COM kadmin: add_principal krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM Enter password for principal "krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM": Re-enter password for principal "krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM": Principal "krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM" created. quit
That means that the A realm will trust the B principal. To create a bidirectional trust, then create principals going the reverse way. Create a principal for the A realm in the B realm, using kadmin.
# kadmin -r B.EXAMPLE.COM kadmin: add_principal krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM Enter password for principal "krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM": Re-enter password for principal "krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM": Principal "krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM" created. quit
Use the get_principal command to verify that both entries have matching key version numbers (kvno values) and encryption types.
It is also possible to reduce the number of hops and represent very complex trust flows by explicitly defining the flow. The [capaths] section of the /etc/krb5.conf file defines the trust flow between different realms.
The format of the [capaths] section is relatively straightforward: there is a main entry for each realm where a client has a principal, and then inside each realm section is a list of intermediate realms from which the client must obtain credentials.
[capaths] A.EXAMPLE.COM = { B.EXAMPLE.COM = . C.EXAMPLE.COM = B.EXAMPLE.COM D.EXAMPLE.COM = B.EXAMPLE.COM D.EXAMPLE.COM = C.EXAMPLE.COM }
With credentials from Realm A, the client obtains a krbtgt/A@A ticket from the KDC of Realm A. Using this ticket, the client then asks for the krbtgt/B@A ticket.
The krbtgt/B@A ticket issued by the KDC of Realm A is a cross-realm ticket granting ticket. It allows the client to ask the KDC of Realm B for a ticket to a service principal of Realm B.
With the krbtgt/B@A ticket, the client asks for the krbtgt/C@B cross-realm ticket.
With the krbtgt/C@B ticket issued by the KDC of Realm B, the client asks for the krbtgt/D@C cross-realm ticket.
With the krbtgt/D@C ticket issued by the KDC of Realm C, the client asks the KDC of Realm D for a ticket to a service principal in Realm D.
After this, the credentials cache contains tickets for krbtgt/A@A, krbtgt/B@A, krbtgt/C@B, krbtgt/D@C, and service/hostname@D. To obtain the service/hostname@D ticket, it was required to obtain the three intermediate cross-realm tickets.
Created 11-09-2017 09:21 AM
We are using standard MIT setup. Its just that we have 3 KDC servers nothing else defers in the configuration process. In fact i created the trust setup from the beginning and its working from Cluster A. Since Cluster B doesn't have hdfs, we cant access Cluster A hdfs with below error.
cabig02@srvbdadvlsk32:~$ hdfs dfs -ls hdfs://srvbdadvlsk21.devoperational1.xxxxxxx.pre.corp:8020/
ls: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
Warning: fs.defaultFS is not set when running "ls" command.
cabig02@srvbdadvlsk32:~$
Created 11-08-2017 06:39 PM
Have you kinit before running the command? The debug log showed that you are running as a non-kerberos user root.
Can you run kdestroy/kinit/klist and ensure your local ticket cache has a valid Kerberos ticket before running the hdfs command?
17/11/08 11:10:48 DEBUG security.UserGroupInformation: hadoop login commit 17/11/08 11:10:48 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root 17/11/08 11:10:48 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: root" with name root 17/11/08 11:10:48 DEBUG security.UserGroupInformation: User entry: "root" 17/11/08 11:10:48 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
Created 11-15-2018 10:46 AM
I got same error, as below
Caused by: GSSException: No valid credentials provided (Mechanism level: Fail to create credential. (63) - No service creds) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 41 more Caused by: KrbException: Fail to create credential. (63) - No service creds at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:162) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693)
And i am able to resolve it by deleting existing krbtgt prinicipals and recreating krbtgt cross realm prinicipals on both clusters with same password. the password must be same for these prinicipals on both KDC's.
krbtgt/A.COM@B.COM
krbtgt/B.COM@A.COM