Cloudera Community
rimm
user rank
New Contributor
My Accepted Solutions
Show All

Re: small input split size in hadoop2.2

by New Contributor in Support Questions
‎05-07-2014 04:09 AM
‎05-07-2014 04:09 AM
Hi, I downloaded hadoop src  2.3 (from http://mir2.ovh.net/ftp.apache.org/dist/hadoop/common/hadoop-2.3.0/), compile it to run under 64bits, and used the following method for setting the input split size     public static void setMaxInputSplitSize(Job job,                                           long size) {     job.getConfiguration().setLong(SPLIT_MAXSIZE, size);   }     best ... View more

small input split size in hadoop2.2

by New Contributor in Support Questions
‎04-30-2014 07:20 AM
‎04-30-2014 07:20 AM
Hi, i am using hadoop 2.2, and don't know how to set max input split size I would like to decrease this value, in order to create more mappers I tried updating yarn-site.xml, and but it does not work indeed, with hadoop 2.2 /yarn, none the following settings has an effet input split size <property> <name>mapreduce.input. fileinputformat.split.minsize</name> <value>1</value> </property> <property> <name>mapreduce.input.fileinputformat.split.maxsiz e</name> <value>16777216</value> </property> <property> <name>mapred.min.split.size</name> <value>1</value> </property> <property> <name>mapred.max.split.size</name> <value>16777216</value> </property> best ... View more
Contact Me
Online
Offline
Last Visited
‎05-08-2014 09:41 AM
Community Statistics
Member Since ‎04-30-2014 07:11 AM
Last Visited ‎05-08-2014 09:41 AM
Posts 2