i,
I have been trying to do a word count programme, which emmits only 1 key value , ie the last key value pair in the input file using wordcount mapreduce programme.
Here is the content of the input file in a directory :
a.txt :
====
f f g h
i i j k
l l m r
f f h h
Content of b.txt
========
r r g h
h h m m
c c b b
d d r f
O/p should be :
r 4
Here is my sample mapper code & reducer code for simple word count. Can anyone tell me what changes should I make to get th o/p like above :
Mapper code:
--------------------
public class WcMapper extends Mapper<LongWritable,Text,Text,IntWritable>{
private static final IntWritable one= new IntWritable(1);
private final Text word=new Text();
public void map(LongWritable key,Text value, Context context
) throws IOException, InterruptedException
{
StringTokenizer st =new StringTokenizer(value.toString());
while(st.hasMoreTokens()){
word.set(st.nextToken());
context.write(word, one);
}
}
}
Reducer code :
---------------------
public void reduce(Text key,Iterable<IntWritable> values, Context context ) throws IOException, InterruptedException{
int sum=0;
for(IntWritable value:values){
sum+= value.get();
}
context.write(key, new IntWritable(sum));
}
}
Driver code::
-----------------------
public class WcDriver extends Configured implements Tool{
public static void main(String[] args) throws Exception {
int status = ToolRunner.run(new WcDriver(), args);
System.exit(status);
}
@Override
public int run(String[] args) throws Exception {
Configuration c1=new Configuration();
Job j1= new Job(c1,"woc");
j1.setJarByClass(WcDriver.class);
j1.setMapperClass(WcMapper.class);
j1.setReducerClass(WcReducer.class);
j1.setInputFormatClass(TextInputFormat.class);
j1.setOutputFormatClass(TextOutputFormat.class);
j1.setOutputKeyClass(Text.class);
j1.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(j1, new Path(args[0]));
FileOutputFormat.setOutputPath(j1, new Path(args[1]));
FileSystem fs = FileSystem.newInstance(c1);
if (fs.exists(new Path(args[1]))) {
fs.delete(new Path(args[1]), true);
}
return j1.waitForCompletion(true) ? 0 : 1;
}
}
Appreciate all help.Please help....
