Cloudera Labs
Provide feedback on Cloudera Labs
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kerberos tickets failing to renew through java code for long running jobs

Kerberos tickets failing to renew through java code for long running jobs

Contributor

Hi,

 

We are unable to renew kerberos user tickets from keytab using java code, while it's working with "kinit -R"

code:

 

UserGroupInformation loginUser = UserGroupInformation.getLoginUser(); loginUser.checkTGTAndReloginFromKeytab();

 

 Please help me out.

7 REPLIES 7

Re: Kerberos tickets failing to renew through java code for long running jobs

Champion

@RakeshE

 

you don't need to renew kerberos ticket from java code instead you can create a shell script with kinit and call it from a cron job. you can also schedule this cron once in 24 hrs or 23.59 hrs (based on your ticket validity period). so that your long running job will run without any issue

Re: Kerberos tickets failing to renew through java code for long running jobs

Contributor

Thanks for your reply @saranvisa. But we are running some jobs through Java code and we should not get the Kerberos login tickets from os level as the different Kerberos users will log into the severs os level.So we are trying to login Kerberos user with keytab through Java for a specific job

Re: Kerberos tickets failing to renew through java code for long running jobs

Champion

@RakeshE

 

In general the option that i've mentioned is the default method. You have to work with your hadoop/unix admin to setup a cron job for your key tab.

 

I don't think controlling kerberos ticket from java code is a good option. Because you have to do this for all your jobs... it is not recommended.  forget about different kerbeors users as long as your team (or) batchid has a dedicated keytab file but if the different kerbeors user is also using your keytab then the ticket validity is common for both of you

Highlighted

Re: Kerberos tickets failing to renew through java code for long running jobs

New Contributor

I'm running into a similar problem, but only in regards to Data At Rest Encryption (DARE).  All other HDFS operations work perpetually and tickets renew as needed.

 

With DARE, everything seems to be set up correctly and works transparently through our app for about an hour, then all we get are "Execution of 'abc.csv' failed. Error details: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)" errors.

 

I thought this might be related to HADOOP-12559 and/or HADOOP-10786 but we upgraded our test environment to CDH 5.8.5 and the problem persists.

 

Manual kinit does not seem to help (and I see valid tickets for our app and for hdfs).

 

Restarting our app seems to reset everything, but I can find no explicit kerberos login that would account for that.

 

My best guess is that there is some principal (possibly HTTP/ourserver.com@REALM.com ?) that needs to renew so that it can validate against the KMS, but doesn't.  I tried manually kinit-ing the HTTP principal on the cm server, but to no avail.

 

An alternate possibility is that something else is failing and the tgt error is a red herring, but the timeout aspect inclines me to think it's a kerberos issue.

 

Any help appreciated!!

Re: Kerberos tickets failing to renew through java code for long running jobs

New Contributor

See my update here

Re: Kerberos tickets failing to renew through java code for long running jobs

New Contributor

@RakeshE @Jeff_M  , I am having the same requirement . Can you please help me out with this , if you got any solution .

Re: Kerberos tickets failing to renew through java code for long running jobs

New Contributor

I am also facing the same issue. I've similar requirment to renew the ticket from Java application. 

 

Sample exception after ticket expiry: 

Removed and destroyed the expired Ticket
Destroyed KerberosTicket

 

 

Kerberos ticket lifetime is set to 24 hours with krb5.conf and before it expires I need to renew the ticket. 

 

Please provide the way to renew the ticket programatically from java application. 

Don't have an account?
Coming from Hortonworks? Activate your account here