- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to override 'mapred.user.jobconf.limit' value in driver class with value set in mapred-site.xml?
- Labels:
-
MapReduce
Created on ‎05-27-2015 01:57 PM - edited ‎09-16-2022 02:30 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have mapreduce job which requires more than current value of 10mb which is set in mapred-site.xml. I want to use temporarily increased value for my current job. Can I set it in the driver class, like 'job.setXXXXXX()' ? And also I dont want to use -D option at command line. Any suggestions ?
Created ‎05-27-2015 09:20 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
limit, and cannot be overridden at a per-job level. You will need to
raise it on the JobTracker config and restart the JobTracker to set it
into effect.
Created ‎05-27-2015 09:36 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created ‎05-27-2015 09:20 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
limit, and cannot be overridden at a per-job level. You will need to
raise it on the JobTracker config and restart the JobTracker to set it
into effect.
Created ‎05-27-2015 09:32 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I know that it could be set in jobtracker but, I am trying to set it at job level to avoid jobtracker restart. So, as I understand it is not at all possible to set it at job level ? Interesting. Thanks Harsh.
Created ‎05-27-2015 09:36 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created ‎05-27-2015 09:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Harsh, appreciate it.
