Getting error while doing distcp with two secure cluster.

[#1]

We have two secure cluster with kerbores. While doing distcp getting following error.

Error:-

Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - UNKNOWN_SERVER) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ... 38 more Caused by: KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:311) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:449) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ... 41 more Caused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66) at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55)

[#2]

Hi @Ashnee Sharma

As the log says that the provided credentials were unable to be identified.

Causes may fall under any of the below:

• The user set the $KRB5_CONFIG environment variable to something other than the default value of /etc/krb5.conf. The HDFS client will not source the $KRB5_CONFIG from the user's shell.
• /etc/gphd/hadoop/conf/hdfs-site.xml does not have the correct Kerberos configuration for the namenode HDFS principal.
• The DNS does not resolve the correct Fully Qualified Domain Name.
• The Kerberos client libs version do not match the server.
• Kerberos encryption defaults differ between the client and the KDC.

Refer the link for the solution on how to overcome this.

https://discuss.pivotal.io/hc/en-us/articles/202210763-The-Secure-HDFS-Error-No-valid-credentials-pr...

Hope it help in solving your issue.

[#3 · Accepted solution]

@Bala Vignesh N V

I have found the acurate issue. Issue was kerberos cross realm. I have configured cross realm:-

https://community.hortonworks.com/articles/18686/kerberos-cross-realm-trust-for-distcp.html

[#4]

@Ashnee Sharma

Good article. Thanks for sharing!