Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

Unable to upload new files to encrypted zone in HDFS


We have an encryption zone set up around a file structure for an application. We are trying to load the files into hdfs in this encrypted zone but we continue to get the following error:

GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)

I have a valid Kerberos ticket and we opened up the KMS acls to allow all users to DECRYPT_EEK just to make sure we did not have an ACL setting wrong


Does anybody else have experience with this issue? 


Thanks I'm certain you're hitting the same error as HADOOP-12559, given the
AuthenticationException is coming at write-time, and from the client
package that's used for HTTP work - indicating that the NN is unable to
contact the KMS.

You'll also likely observe this error only much after a NameNode restart
period (but that it works immediately after NN restart), and that it may go
away after one day or so, only to return again, which is inline with
behaviour within the NameNode.

The bug-fix update of 5.5.x or any minor upgrade to the newer releases
should solve this up.

View solution in original post



FWIW I seem to have found a solution.


I had added  a call to 


but it hadn't worked.


Later in debugging I found that that call was trying to renew the Proxy User, not the underlying principal.


I changed the call so that it would get the principal's ugi and call the same method on that and now it seems to work.


There are still outstanding questions, though, if anyone cares to investigate further:


  • Why was this only necessary for D.A.R.E. ?  All other ops (hdfs, Hive, yarn, etc.) continued working and renewing krbtgt's perpetually
  • Was the upgrade of CDH needed or would it have continued working with the older version?