Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Changing user history location on MR2 jobs

Solved Go to solution

Changing user history location on MR2 jobs

New Contributor

Hi,

 

I'm using MR2 with CDH5 and I noticed that the user history location (on HDFS) for completed jobs is /user/history/done

 

I'm trying to automate some jobs and then analyze their job history, so it could be useful for me to set a different location per job.

 

How can I do it?

 

Thanks,

Elad I

Mellanox Technlogies

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Changing user history location on MR2 jobs

Super Collaborator

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.

 

Wilfred

2 REPLIES 2

Re: Changing user history location on MR2 jobs

New Contributor

Update:

I found the parameter: mapreduce.jobhistory.done-dir

With it I was able to set the root directory for done history files

But that doesn't completely solve my problem

Right now the directory structure is /rootdir/year/month/day/000000/jhist-file

I'd like to drop the whole year/month/day/000000 part. Is that possible?

Highlighted

Re: Changing user history location on MR2 jobs

Super Collaborator

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.

 

Wilfred

Don't have an account?
Coming from Hortonworks? Activate your account here