Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Is it possible to dynamically set map and reduce memory for a Mapreduce job?

avatar
Rising Star
 
1 ACCEPTED SOLUTION

avatar

@Phoncy Joseph

The configs can be changed when you are submitting the job and not while the job is running. Below are the configs to change them while submitting the job.

-Dmapreduce.map.memory.mb=2048 -Dmapreduce.reduce.memory.mb=2048

View solution in original post

5 REPLIES 5

avatar

@Phoncy Joseph

The configs can be changed when you are submitting the job and not while the job is running. Below are the configs to change them while submitting the job.

-Dmapreduce.map.memory.mb=2048 -Dmapreduce.reduce.memory.mb=2048

avatar
Rising Star

Thanks for the response. I wanted to know if the memory assignment could be done, without providing these while submitting jobs.

avatar

@Phoncy Joseph There are only 3 ways as of now:

- Global configs which are in mapred-site.xml

- Passing through command line while submitting the job

- Hardcode in the code(although this is not dynamic).

avatar
Rising Star

Thank you

avatar

@Phoncy Joseph You're welcome. Feel free to accept the answer if this helps you.