Created 06-27-2017 09:54 PM
i am setting this parameter to use 32 mappers but its still using one
hive> set map.reduce.tasks=32
> explain select sum(amnt) from lanetxn; OK Plan not optimized by CBO. Vertex dependency in root stage Reducer 2 <- Map 1 (SIMPLE_EDGE) Stage-0 Fetch Operator limit:-1 Stage-1 Reducer 2
Created 06-28-2017 04:26 AM
@Sami Ahmad try hive> set mapred.map.tasks = 32;
Created 06-28-2017 01:29 PM
still the same issue , its still using only one map task and 2 reduce tasks.
Created 07-01-2017 03:44 PM
Maximum number of mappers will be bound by the number of splits calculated at split generation time.
These settings impact split calculation:
Splits are grouped at the Tez layer based on these settings:
If you want more mappers you can tune all these settings down. Note this will not guarantee lower latency, especially on small clusters.