Reply
New Contributor
Posts: 5
Registered: ‎06-19-2014
Accepted Solution

Changing user history location on MR2 jobs

Hi,

 

I'm using MR2 with CDH5 and I noticed that the user history location (on HDFS) for completed jobs is /user/history/done

 

I'm trying to automate some jobs and then analyze their job history, so it could be useful for me to set a different location per job.

 

How can I do it?

 

Thanks,

Elad I

Mellanox Technlogies

New Contributor
Posts: 5
Registered: ‎06-19-2014

Re: Changing user history location on MR2 jobs

Update:

I found the parameter: mapreduce.jobhistory.done-dir

With it I was able to set the root directory for done history files

But that doesn't completely solve my problem

Right now the directory structure is /rootdir/year/month/day/000000/jhist-file

I'd like to drop the whole year/month/day/000000 part. Is that possible?

Highlighted
Cloudera Employee
Posts: 290
Registered: ‎01-16-2014

Re: Changing user history location on MR2 jobs

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.

 

Wilfred

Announcements