Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to run multiple jobs on kerberos cluster after logging off from Hadoop server

Highlighted

How to run multiple jobs on kerberos cluster after logging off from Hadoop server

Rising Star

Hi,

I am submitting a shell script that run multiple Hadoop jobs one after another.

I do kinit

Submit the script and logoff from the server.

Problem here is:

Only the first job executes successfully and rest of teh jobs fails with Kerberos ticket not found error.

I understand that once I logout from server the Kerberos credential get lost.

The first job is successful because namenode stores credential as HDFS Delgation token.

How can I use the credentials created before I logout to execute jobs that start running after I logout.

3 REPLIES 3
Highlighted

Re: How to run multiple jobs on kerberos cluster after logging off from Hadoop server

@Venkata Sridhar Gangavarapu

Can you try oozie or azkaban tools which can take care of kerberos ticket renewal.

Re: How to run multiple jobs on kerberos cluster after logging off from Hadoop server

Contributor

Best is to leverage tools like Oozie which will handle ticket renewal.

Alternatively, you can try running your script file with nohup command, so that logout doesn't affect the script from running

https://www.cyberciti.biz/tips/nohup-execute-commands-after-you-exit-from-a-shell-prompt.html

Highlighted

Re: How to run multiple jobs on kerberos cluster after logging off from Hadoop server

Rising Star

Hi Vijay and sagar,

Thanks for your inputs.

Can you please suggest me the module in Oozie that does the renewal work.

Don't have an account?
Coming from Hortonworks? Activate your account here