- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How Tez decides the Reducers order and spins the containers
Created 10-08-2020 05:50 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In My HDP 3.1.5 cluster, Tez is not following the reducer order in the VERTICES (shown in the first pic). I am interested in knowing how Tez decides the order and number of containers in a VERTICES. I can even see single-digit containers in a few VERTICES. Resources are free on the cluster but it has very few containers running.
Created 10-20-2020 01:10 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can check the explain plan . It provided the order of the execution.
It is not a rule that certain reucers would start after all the mapper phase.
The ceratin mapper phase depends on the results from the reducers too. You can check the explain plan .
If you want map join then you can set property map.auto.conevrt.join =true
