Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

It's fine this number of maps?

avatar
Expert Contributor

Hello I have a hive table splitted in 200 files

I was waiting to see 200 map task but counters only show 10 map task, why is this happening?

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).

Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.

View solution in original post

3 REPLIES 3

avatar
Super Collaborator

Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).

Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.

avatar
Expert Contributor

Thank was just this, mapreduce.input.fileinputformat.split.maxsize was set to 256000000.

avatar
Super Collaborator

Great to hear. Thanks for the feedback.