Reply
Highlighted
New Contributor
Posts: 5
Registered: ‎06-19-2014
Accepted Solution

Changing user history location on MR2 jobs

Hi,

 

I'm using MR2 with CDH5 and I noticed that the user history location (on HDFS) for completed jobs is /user/history/done

 

I'm trying to automate some jobs and then analyze their job history, so it could be useful for me to set a different location per job.

 

How can I do it?

 

Thanks,

Elad I

Mellanox Technlogies

New Contributor
Posts: 5
Registered: ‎06-19-2014

Re: Changing user history location on MR2 jobs

Update:

I found the parameter: mapreduce.jobhistory.done-dir

With it I was able to set the root directory for done history files

But that doesn't completely solve my problem

Right now the directory structure is /rootdir/year/month/day/000000/jhist-file

I'd like to drop the whole year/month/day/000000 part. Is that possible?

Cloudera Employee
Posts: 314
Registered: ‎01-16-2014

Re: Changing user history location on MR2 jobs

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.

 

Wilfred

Announcements

Our community is getting a little larger. And a lot better.


Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.