I have a script that should run in a cron job and should be authenticated with hdfs user through kerberos.
To run the script outside the cron job, from the shell, I execute the following commands:
sudo -i kinit -V -k -t /etc/security/keytabs/hdfs.headless.keytab hdfs callMyScriptWithParams
The above commands execute as I needed them to. However, when I call the same set of commands in a cron job, I get the following error
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
I get the same error if I tried to run the same commands from the shell but not through (Using my current user) the root as below
sudo kinit -V -k -t /etc/security/keytabs/hdfs.headless.keytab hdfs #works fine sudo callMyScriptWithParams #throw the error
I tried to create several versions of the cron job, one of them is below (Runs every three minutes for testing purposes)
*/3 * * * * root sudo -i; kinit -V -kt /etc/security/keytabs/hdfs.headless.keytab hdfs; klist; callMyScriptWithParams; klist
I am calling 'klist' to check that I am getting the correct ticket. Klist returen the hdfs user ticket before and after calling my script. Since I have a valid ticket, I am not sure why am I getting the above error. Below is the output when I obtain the ticket:
Using default cache: /run/user/krb5cc/krb5cc_0 Using principal: hdfs@MyRealm Using keytab: /etc/security/keytabs/hdfs.headless.keytab Authenticated to Kerberos v5
and this is an example of a retrieved ticket from 'klist'
Ticket cache: FILE:/run/user/krb5cc/krb5cc_0 Default principal: hdfs@MyRealm Valid starting Expires Service principal 10/17/2016 15:12:01 10/18/2016 15:12:01 krbtgt/MyRealm@MyRealm
If I am retrieving a valid ticket before and after calling myscript, then why am I getting 'Failed to find any Kerberos tgt' error when I call the script? specially that I called the same commands outside the cron job and they worked fine.
P.S. I tried to cron job without the 'sudo -i' as well but I am still getting the same error.
Please check if the user running the cron job has permission to read keytab file.
Following is a relevant post that can help you further.
The user does have permission, when I run klist before and after calling my script I find a valid ticket which means that the cron job was able read the keytab file. I used the link to be able to call multiple commands in the same cron job line. It still does not explain why am I having this error I am afraid :(