- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Number of mapper
Created ‎12-07-2021 03:14 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Our user is running a job and which is a hive query and number of mapper is always 6 and not changing even the data size change. It is a insert query. How do I change number of mappers ? Which parameter determine number of mappers?
Created ‎12-15-2021 02:09 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Our user is running a job and which is a hive query and number of mapper is always 6 and not changing even the data size change. 192168101.win 100001.dev routerlogin.win
It is a insert query. How do I change number of mappers ? Which parameter determine number of mappers?
Thank you.
Created ‎12-16-2021 05:53 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hello Becky
You can either reduce "split max size" in order to gain more mappers
SET mapreduce.input.fileinputformat.split.maxsize;
Or you can try :
Set mapreduce.job.maps=XX
for the second option, you may need to disable map files
hive.merge.mapfiles=false
Let me know if any solution works for you
Good luck
