Support Questions
Find answers, ask questions, and share your expertise

It's fine this number of maps?

Solved Go to solution

It's fine this number of maps?

Expert Contributor

Hello I have a hive table splitted in 200 files

I was waiting to see 200 map task but counters only show 10 map task, why is this happening?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: It's fine this number of maps?

Expert Contributor

Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).

Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.

View solution in original post

3 REPLIES 3

Re: It's fine this number of maps?

Expert Contributor

Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).

Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.

View solution in original post

Re: It's fine this number of maps?

Expert Contributor

Thank was just this, mapreduce.input.fileinputformat.split.maxsize was set to 256000000.

Re: It's fine this number of maps?

Expert Contributor

Great to hear. Thanks for the feedback.