It took me a few moments, but it looks like you are doing something that I also ran into problems with initially, which is to execute a hive command through a shell action. I never found a resolve for this and thought that pursuing that course was going a bit against the purpose of the actions themselves (shell versus hive). I think eventually you could perform some hacks to get it working, it may not be a preferred path. The approach I took to resolve this was to adhere to the practices that Oozie lays out for me, instead of trying to bend it to the way I wanted to do it, which was to create a cool wrapper script that would dynamically execute whatever I passed in. I'm getting a bit off topic on this ,but if you want to do dynamic stuff like that, it might be better to dynamically create the hql out in a file, then dynamically create the workflow content of an already existing statically-called workflow next (again... I'm digressing so I'll stop) Try using a Hive action and create a separate hiveQL file with the SQL that you are wanting to perform. Otherwise, it looks like you were hitting on all of the right things to include, like the hive-site.xml, credentials, etc. <global>
<job-xml>${wfJobSite}</job-xml>
</global>
[...]
<action name="hive-action" cred="hcat">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<script>/directory-path/example.hql</script>
</hive>
<ok to="End"/>
<error to="Kill"/>
</action>
[...] I don't know for sure what is happening behind the scenes when Oozie executes a shell script and you try to run a Hive command, but my hypothesis is that when the script executes out on a data node, it loses the context of the credentials and details you pass in, and unless you expressly leverage those somehow in your script, it results in the script not executing hive actions properly. A friend of mine was able to get a script to execute sqoop but only after he copied JARs/files out to every data node. I don't recommend trying to bend Hadoop to your will like that, but instead try to leverage the tools as they were designed and perhaps commit to the tools to eventually bend them to your will. 🙂 [just my thoughts]
... View more
