Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to override 'mapred.user.jobconf.limit' value in driver class with value set in mapred-site.xml?

avatar
Contributor

I have mapreduce job which requires more than current value of 10mb which is set in mapred-site.xml. I want to use temporarily increased value for my current job. Can I set it in the driver class, like 'job.setXXXXXX()' ? And also I dont want to use -D option at command line. Any suggestions ?

Engineer
2 ACCEPTED SOLUTIONS

avatar
Mentor
If this is an MR1 question, then the config mentioned is a server-side
limit, and cannot be overridden at a per-job level. You will need to
raise it on the JobTracker config and restart the JobTracker to set it
into effect.

View solution in original post

avatar
Mentor
Yes, it is an administrative limit, so it cannot be changed per job.

View solution in original post

4 REPLIES 4

avatar
Mentor
If this is an MR1 question, then the config mentioned is a server-side
limit, and cannot be overridden at a per-job level. You will need to
raise it on the JobTracker config and restart the JobTracker to set it
into effect.

avatar
Contributor

I know that it could be set in jobtracker but, I am trying to set it at job level to avoid jobtracker restart. So, as I understand it is not at all possible to set it at job level ? Interesting. Thanks Harsh.

 

Engineer

avatar
Mentor
Yes, it is an administrative limit, so it cannot be changed per job.

avatar
Contributor

Thanks Harsh, appreciate it.

Engineer