Support Questions

Find answers, ask questions, and share your expertise

How Tez decides the Reducers order and spins the containers

avatar
Explorer

In My HDP 3.1.5 cluster, Tez is not following the reducer order in the VERTICES (shown in the first pic). I am interested in knowing how Tez decides the order and number of containers in a VERTICES. I can even see single-digit containers in a few VERTICES. Resources are free on the cluster but it has very few containers running. 

santoshh_0-1602161140129.png

 

 

1 REPLY 1

avatar
Guru

You can check the explain plan . It provided the order of the execution.

 

It is not a rule that certain reucers would start after all the mapper phase.

 

The ceratin mapper phase depends on the results from the reducers too. You can check the explain plan .

 

If you want map join then you can set property map.auto.conevrt.join =true