Our user is running a job and which is a hive query and number of mapper is always 6 and not changing even the data size change. It is a insert query. How do I change number of mappers ? Which parameter determine number of mappers?
The following parameters control the number of mappers for splittable formats with Tez:
set tez.grouping.min-size=16777216; -- 16 MB min split
set tez.grouping.max-size=1073741824; -- 1 GB max split
Adjust the above values to best suit your data file size to avoid file split grouping leading to increased number of mappers.
If you still don't see number of mappers increased and hive.input.format is set to “org.apache.hadoop.hive.ql.io.CombineHiveInputFormat”, you may need to adjust below properties as well
Please note that data locality w.r.t nodes also plays roles in determining, for more information please refer to the below references