Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

It's fine this number of maps?

Solved Go to solution

It's fine this number of maps?

Expert Contributor

Hello I have a hive table splitted in 200 files

I was waiting to see 200 map task but counters only show 10 map task, why is this happening?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: It's fine this number of maps?

Expert Contributor

Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).

Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.

3 REPLIES 3

Re: It's fine this number of maps?

Expert Contributor

Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).

Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.

Re: It's fine this number of maps?

Expert Contributor

Thank was just this, mapreduce.input.fileinputformat.split.maxsize was set to 256000000.

Highlighted

Re: It's fine this number of maps?

Expert Contributor

Great to hear. Thanks for the feedback.