I have tried with the following parms hive.tez.auto.reducer.parallelism=true; hive.tez.min.partition.factor=0.25 hive.tez.max.partition.factor=2.0 set hive.exec.reducers.bytes.per.reducer = 134217728; My output is of size 2.5 GB (2684354560 bytes) and based on the formula given above, i was expecting max(1,min(1099,2684354560/ 134217728))*2 = max(1,min(1099,20))*2 = max(1,20)*2 = 40 reducers. but my query was assigned only 5 reducers, i was curious why? and are there any other parameters that can reflect the no. of reducers. below is the query that i am using : truncate table target_tab ;
INSERT INTO TABLE target_tab
SELECT * FROM src_tab WHERE 1=1 ORDER BY a, b,c
... View more
