Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Changing user history location on MR2 jobs

avatar
Explorer

Hi,

 

I'm using MR2 with CDH5 and I noticed that the user history location (on HDFS) for completed jobs is /user/history/done

 

I'm trying to automate some jobs and then analyze their job history, so it could be useful for me to set a different location per job.

 

How can I do it?

 

Thanks,

Elad I

Mellanox Technlogies

1 ACCEPTED SOLUTION

avatar
Super Collaborator

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.

 

Wilfred

View solution in original post

2 REPLIES 2

avatar
Explorer

Update:

I found the parameter: mapreduce.jobhistory.done-dir

With it I was able to set the root directory for done history files

But that doesn't completely solve my problem

Right now the directory structure is /rootdir/year/month/day/000000/jhist-file

I'd like to drop the whole year/month/day/000000 part. Is that possible?

avatar
Super Collaborator

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.

 

Wilfred