Created 09-15-2016 04:36 PM
Hello I have a hive table splitted in 200 files
I was waiting to see 200 map task but counters only show 10 map task, why is this happening?
Created 09-15-2016 04:49 PM
Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).
Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.
Created 09-15-2016 04:49 PM
Hive will combine input splits by default (https://issues.apache.org/jira/browse/HIVE-2245).
Some tips on how to control this are here. Look at the mapred.max.split.size and mapred.min.split.size properties.
Created 09-16-2016 10:12 AM
Thank was just this, mapreduce.input.fileinputformat.split.maxsize was set to 256000000.
Created 09-16-2016 01:31 PM
Great to hear. Thanks for the feedback.