Support Questions
Find answers, ask questions, and share your expertise

Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

Solved Go to solution
Highlighted

Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

Rising Star

Hi Everyone,

I came across a Kerberos cache issue and wanted to share and possible have more ideas.

One of our clients have RedHat IDM (supported version of freeIpa) and when you install sssd along with krb5 by IDM the default cache setting is 'KEYRING' than 'File'

You will still be able to get the tickets but will have GSSException error on Cluster.

KEYRING persistent cache setting works with many applications however not with HADOOP Cluster, I'm sure HDO engineering team must be looking into this for a solution, since KEYRING is the future for kerberos cache and the stuff you can do with it like keylist and etc.

To solve the errors you can comment out the "default_ccache_name=KEYRING....." on krb.conf or change it to "default_ccache_name = FILE:/tmp/krb5cc_%{uid}"

Logout and log in again - destroy the previous tickets and you should have something like "Ticket cache: FILE:/tmp/krb5cc_" in your klist output.

If you still see KEYRING PERSISTENT, kill all the running sessions of the user having the problem and restart SSSD service.

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

@mkataria, thanks for this

"I'm sure HDO engineering team must be looking into this for a solution, since KEYRING is the future for kerberos cache and the stuff you can do with it like keylist and etc."

Hadoop uses the Kerberos libraries that come with the JVM. That's pretty problematic, because we have to delve into the private com.sun. and com.ibm. internal classes to get some stuff done, and that tends to break across major (and sometimes minor) releases. If we stay with the Oracle libraries, we're stuck with whatever they implement, which, as you've noticed, is pretty limited and out of date.

You may be interested to hear of Apache Kerby, an ongoing project to have a pure Java Kerberos Domain Controller and, importantly for production Hadoop —a modern Kerberos client library, with CPU-acceleration of encryption when available (i.e. latest intel parts, as Hadoop already does for HDFS encryption), and broader protocol support. I don't think its ready for production use yet, and nobody has —yet— gone beyond looking at what it would take to switch Hadoop over to it. But be assured: people are looking at it.

Finally, what was the error message you got? Was it the classic "unable to find TGT for user" kind of text, or something new? I'm trying to build a list list of common Kerberos error messages —adding a new message or a new possible cause of an existing message is something else to put in here.

View solution in original post

8 REPLIES 8
Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

@mkataria perhaps you would consider posting this as an article instead of a question? It would be a useful article because others have run into this in the past (see here)

Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

Rising Star

Thanks @Ali Bajwa, posted as article

Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

@mkataria, thanks for this

"I'm sure HDO engineering team must be looking into this for a solution, since KEYRING is the future for kerberos cache and the stuff you can do with it like keylist and etc."

Hadoop uses the Kerberos libraries that come with the JVM. That's pretty problematic, because we have to delve into the private com.sun. and com.ibm. internal classes to get some stuff done, and that tends to break across major (and sometimes minor) releases. If we stay with the Oracle libraries, we're stuck with whatever they implement, which, as you've noticed, is pretty limited and out of date.

You may be interested to hear of Apache Kerby, an ongoing project to have a pure Java Kerberos Domain Controller and, importantly for production Hadoop —a modern Kerberos client library, with CPU-acceleration of encryption when available (i.e. latest intel parts, as Hadoop already does for HDFS encryption), and broader protocol support. I don't think its ready for production use yet, and nobody has —yet— gone beyond looking at what it would take to switch Hadoop over to it. But be assured: people are looking at it.

Finally, what was the error message you got? Was it the classic "unable to find TGT for user" kind of text, or something new? I'm trying to build a list list of common Kerberos error messages —adding a new message or a new possible cause of an existing message is something else to put in here.

View solution in original post

Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

Rising Star

@stevel thanks for the explanation upcoming Kerby, we had the same error message which you have already documented however a different cause 'may be unsupported cache type" the error was

No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt

we commented out "default_ccache_name = KEYRING:persistent:%{uid}" and it fixed.

Thanks Mayank

Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

Explorer

Looks like that freeipa-client on CentOS6.x is not adding the default_ccache_name entry in the /etc/krb5.conf while it adds it in CentOS7, and the KEYRING: thing makes it all go bonkers indeed.

Thanks a lot for the pointer!!!!

Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

THANK YOU! I barely can imagine how many nights you wasted on this. This vicious issue appeared suddenly when we installed FreeIPA client on centos 7 nodes. Whereas, on CentOS 6.7, the KEYRING was absent...

Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

Please also note the exact same error can be caused by missing Oracle JCE jar files. It turns out openJDK ships JCE policy jar files compatible with HDP, but not the Oracle JVM. Hence, the extensions should be installed (even if the $JAVA_HOME/lib/security folder already contain the jars, they may not be then proper ones). I learnt this the hard way.

Highlighted

Re: Kerberos Cache in IPA /RedHat IDM (KEYRING) SOLVED!!

New Contributor

Using KEYRING was a state of art at the moment Kerberos was bundled for RHEL7. However moving forward into the world of containers using KEYRING becomes a challenge thus SSSD is building internal ticket cache that will be supported by the system Kerberos libraries. So in general the recommendation nowadays is to use native OS Kerberos libraries, they are most recent and will provide latest functionality and experience.