Change oozie-oozi folder location

[#1]

I was trying to run a mapreduce job example in hue.

When I submitted a pig job through oozie, I see a folder was created on HDFS as "/user/<my_user_name>/oozie-oozi". And there is a "pig-pig" folder in it which includes an output folder and an action.xml file. This "oozie-oozi" folder seems like a task's attempts working directory, and only has contents when job is running and got cleaned up when job is done.

Can anyone please help me understand what this "oozie-oozi" folder is? And how can I configure to change the location of this folder?

[#2 · Accepted solution]

The oozie-oozi is the system-id of your Oozie server, and the contents of such a directory on HDFS carries job running data pertaining to your various actions under it.

This path is not configurable at the moment, however, as it roots itself on the user's home directory on the FS, which on HDFS translates to /user/<my_user_name>.

Is the presence of the directory causing a problem? The directory is cleaned up at the end typically.

[#3]

Thanks Harsh, the answer is really helpful. I can better understand this directory now.

[#4]

I'm running oozie HA 5.2.1 on EMR and I have an issue with this temporary directory. I have a workflow which has start node -> action node -> end node. The job start running -> runs for 10-15 minutes -> is initially marked as successful -> is marked as failed. The error is JA008 File or directory not found for the /user/oozie/oozie-oozi/"oozie_job_id"/"ActionName"--java directory. I'm running in parallel a script which looks at this directory as well as the state of the workflow and while running this directory contains action.xml and launcher.xml file -> before marking it as SUCCEDED it adds the action-data.seq file -> after it marks it as succeeded it deletes this directory -> after 1-2 minutes marks it as failed with the error above. It looks like oozie is checking again for that directory after it previously deleted it.

Although this jobs is marked as failed from oozie perspective in reality this job is successful (checking in the RM UI the jobs status and the logs).

This error occurs multiple times a day. Sometimes the workflow is succeeded other times is marked as failed from oozie perspective but in reality is succeeded.

Any idea why this happens (looks like a race condition) ?

[#5]

@StefanSs As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks.