question Changing user history location on MR2 jobs in Archives of Support Questions (Read Only)

Changing user history location on MR2 jobs

eladi — Fri, 16 Sep 2022 09:00:28 GMT

Hi,

I'm using MR2 with CDH5 and I noticed that the user history location (on HDFS) for completed jobs is /user/history/done

I'm trying to automate some jobs and then analyze their job history, so it could be useful for me to set a different location per job.

How can I do it?

Thanks,

Elad I

Mellanox Technlogies

Re: Changing user history location on MR2 jobs

eladi — Thu, 19 Jun 2014 11:27:08 GMT

Update:

I found the parameter: mapreduce.jobhistory.done-dir

With it I was able to set the root directory for done history files

But that doesn't completely solve my problem

Right now the directory structure is /rootdir/year/month/day/000000/jhist-file

I'd like to drop the whole year/month/day/000000 part. Is that possible?

Re: Changing user history location on MR2 jobs

Wilfred — Sat, 21 Jun 2014 12:19:19 GMT

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.

Wilfred