Support Questions
Find answers, ask questions, and share your expertise

question on executing HDFS command on Kerberized cluster through oozie Shell action

question on executing HDFS command on Kerberized cluster through oozie Shell action

Expert Contributor


I am trying to read a file in HDFS using hadoop fs -cat command in oozie Shell action. Mine is a kerberized cluster.

oozie workflow is submitted using my ID, A. The file can be only read using ID B. I am doing kinit -kt using B's keytab.

Inside the shell script, I did kinit -kt and i also did klist. Klist displayed B as default principal and it showed a valid ticket.

But even though the klist shows B's valid ticket, hadoop fs -cat is executed using my ID (A) and not B. This results in insufficient privilege issue. Why is the hadoop fs -cat command not using B's ticket and using my ID ?

The same thing works when I run from linux as individual commands instead of oozie shell action. I login to linux using my ID. klist just shows my principal. I do a kinit for B and now Klist shows B's ticket. and in the same shell (not from oozie, from linux command line) when I issue hadoop fs -cat filename, it displays the content of the file.

Why is this working from linux directly but not working when executed from oozie shell action?

After doing a kinit on a different user, all the hadoop commands seem to be executing using the second user in linux CLI, I thought this is how the oozie shell action would work too. Please help me understand this.

Note: When I login using my ID and do kinit as second user before even submitting the oozie workflow, and submit the oozie workflow after kinit to second user, this seems to work and all actions inside shell action of workflow now seems to exxecuted by second user rather than my ID. and this way it does not create any issues. Please help me understand this as well.