how to set number of map and reduce tasks

[#1]

I am running a query which runs 52 map jobs simultaneously. Due to this my Resource manager container gets filled up completely and consumed up 100%. The query stucks at that point and giving no result.

I want to reduce number of map tasks which runs in parallel.

[#2 · Accepted solution]

You will have to pass runtime parameters to Hive query.

See this post.

https://community.hortonworks.com/questions/905/how-are-number-of-mappers-determined-for-a-query-w.h...

[#3]

@Tajinderpal Singh Also, look at mapreduce.job.reduce.slowstart.completedmaps properties in map-reduce and set this to 0.9. This property will ensure reducers not coming in early and waiting for mappers to complete there by avoiding hung jobs.

[#4]

You can set the run time parameters within hive shell, or pass them through your script, as mentioned by Pranay. Also if you are using Tez, this particular article on how to tune the performance may come in Handy.

https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer....