Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hadoop wordcount mapreduce program giving error

hadoop wordcount mapreduce program giving error

New Contributor

I have written a simple word count java program in hadoop 2.6.0 on Cloudera vm. Following are my three programs present in three different files.

 

WordCountMapper.java file

----------------------------------------------------------------------------------------------------------------------------------------------------

package com.org.hadoop.dev.programs;

import java.io.IOException;
import java.util.StringTokenizer;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Mapper;

public class WordCountMapper extends Mapper<LongWritable,Text,Text,IntWritable>{

public void map(LongWritable key,Text value, Context context) throws IOException, InterruptedException{

Text text = new Text();
String line = value.toString();

StringTokenizer tokens = new StringTokenizer(line);

while(tokens.hasMoreElements()){

String word = tokens.nextToken();
text.set(word);
context.write(text, new IntWritable(1));
}

}

}

 

-----------------------------------------------------------------------------------------------------------------------------------------------------

WordCountReducer.java

-----------------------------------------------------------------------------------------------------------------------------------------------------

package com.org.hadoop.dev.programs;

import java.io.IOException;

import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;

public class WordCountReducer extends Reducer<Text, IntWritable, Text, IntWritable> {

public void reduce(Text text, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException{

int sum = 0;
for(IntWritable value : values){
sum += value.get();
}
context.write(text, new IntWritable(sum));
}

}

------------------------------------------------------------------------------------------------------------------------------------------------------

WordCountDriver.java

------------------------------------------------------------------------------------------------------------------------------------------------------

package com.org.hadoop.dev.programs;

import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.Reducer;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.IntWritable;;

public class WordCountDriver {

public static void main(String[] args) throws Exception{

JobConf job = new JobConf(new Configuration(), WordCountDriver.class);
job.setJarByClass(WordCountDriver.class);

job.setJobName("Word Couner");

FileInputFormat.addInputPaths(job, args[0]);
FileOutputFormat.setOutputPath(job, new Path(args[1]));

job.setMapperClass((Class<? extends Mapper>) (WordCountMapper.class));
job.setReducerClass((Class<? extends Reducer>) WordCountReducer.class);


job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);

}

}

------------------------------------------------------------------------------------------------------------------------------------------------------Error received on runing WordCountDriver.java file

 

Exception in thread "main" java.lang.RuntimeException: class com.org.hadoop.dev.programs.WordCountMapper not org.apache.hadoop.mapred.Mapper
at org.apache.hadoop.conf.Configuration.setClass(Configuration.java:2276)
at org.apache.hadoop.mapred.JobConf.setMapperClass(JobConf.java:1123)
at com.org.hadoop.dev.programs.WordCountDriver.main(WordCountDriver.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

 

------------------------------------------------------------------------------------------------------------------------------------------------------Command used for running this program 

 

hadoop jar /home/cloudera/test/wordCount.jar com.org.hadoop.dev.programs.WordCountDriver /input/file.txt /output

 

------------------------------------------------------------------------------------------------------------------------------------------------

 

Any help is highly appreciated.

Don't have an account?
Coming from Hortonworks? Activate your account here