We have Hortonworks environment and when i am running one hive query(which has multiple join) it is getting hung at reducer level and take 3 hrs to finish some time and some time it get hung.
According to our hadoop support team they said we need to add memory in the queue but they said I need to tell them how much memoy I need to run my query.
So how to calculate how much memory I need to execute my hive query successfully.
When you say it is hung at reducer level,so all the containers take more time or few containers in the reudcer takes lot of time and hung.
There is a data skewness,if few conatimers at reducer level takes time.
You have to re-write the query if there is a data skweness