06-19-2014 03:52 AM
I'm using MR2 with CDH5 and I noticed that the user history location (on HDFS) for completed jobs is /user/history/done
I'm trying to automate some jobs and then analyze their job history, so it could be useful for me to set a different location per job.
How can I do it?
06-19-2014 04:27 AM
I found the parameter: mapreduce.jobhistory.done-dir
With it I was able to set the root directory for done history files
But that doesn't completely solve my problem
Right now the directory structure is /rootdir/year/month/day/000000/jhist-file
I'd like to drop the whole year/month/day/000000 part. Is that possible?
06-21-2014 05:19 AM
No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.