Is it possible to dynamically set map and reduce memory for a Mapreduce job?

[#1 · Accepted solution]

@Phoncy Joseph

The configs can be changed when you are submitting the job and not while the job is running. Below are the configs to change them while submitting the job.

-Dmapreduce.map.memory.mb=2048 -Dmapreduce.reduce.memory.mb=2048

[#2]

Thanks for the response. I wanted to know if the memory assignment could be done, without providing these while submitting jobs.

[#3]

@Phoncy Joseph There are only 3 ways as of now:

- Global configs which are in mapred-site.xml

- Passing through command line while submitting the job

- Hardcode in the code(although this is not dynamic).

[#4]

Thank you

[#5]

@Phoncy Joseph You're welcome. Feel free to accept the answer if this helps you.