Use Oozie shell action to write in a local file

[#1]

hi, I need my job.properties to be edited after each time a sqoop action is done through a oozie coodrinator.

I have added a shell action after sqoop action in my workFlow.

my job.properties is not in hdfs and is located in node2, where I run oozie job. (i have 5 nodes),

but the shell faces 'permission denied' error, and sometimes it says "no such file or directory'. i moved it to hdfs but the same thing happened.

has anyone done such a thing?

I execute the shell commands locally and they work correctly, but somehow when it's running distributed I can't control permissions.

[#2 · Accepted solution]

Oozie executes the Shell action script file from a launcher job. The launcher can be allocated to any node in the hadoop cluster, So the error "no such file or directory' means that, launcher job node is different than node2 where you have stored the file. For the 'permission denied' error, please check the proper write permission set for the file, as user can be different while modifying it from oozie shell action. Though it is not advised to modify a local file from a Oozie action, as launched action can be on any node in the cluster.

Once you move the file to HDFS, use the proper HDFS commands. You can not modify the file in HDFS, you need to get the file and after modification, put (store) the file again.