Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

what is mapreduce.task.io.sort.mb relate to?

avatar
Explorer

map.memory = 2G map.opt=2048M io.sort.mb=2047M will cause ERROR JAVA HEAP SIZE

map.memory = 2G map.opt=3096M io.sort.mb=2047M is fine;

why?

1 REPLY 1

avatar
Super Collaborator

What do you have following property set to "yarn.scheduler.minimum-allocation-mb". when you set map.memory = 2G, you might end up getting a 4GB container with opt 2048.