Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to parse java double into Hadoop map reduce DoubleWritable?

Highlighted

How to parse java double into Hadoop map reduce DoubleWritable?

New Contributor

Hi I am new to Hadoop I am stuck at a point where I do not have any clue what am I doing wrong. I am trying to parse simple double value into DoubleWritable of hadoop but it is giving me 0.0.

0.0 Southwest 
0.0 US Airways 
0.0 United 
0.0 Virgin America 

Below is what I am trying to do:

public class USAirlineComparisionReducer extends Reducer<Text, DoubleWritable, Text, DoubleWritable> {
	private Logger logger = Logger.getLogger(USAirlineComparisionReducer.class);
	@Override
	protected void reduce(Text key, Iterable<DoubleWritable> values, Context context)
			throws IOException, InterruptedException {
		double negativeWords = 0;
		for(DoubleWritable value : values) {
			if(value.get() < 0) {
				negativeWords = negativeWords + 1;
			}
		}
		logger.info(" KEY: " + key + " NEGATIVE WORDS: " +negativeWords);
		context.write(key, new DoubleWritable(negativeWords));
	}
}
Mapper: 
public class USAirlineComparisionMapper extends Mapper<LongWritable, Text, Text, DoubleWritable> {
	private Logger logger = Logger.getLogger(USAirlineComparisionMapper.class);
	private final static DoubleWritable tempResult = new DoubleWritable();
	private Text tempAirline = new Text();
	private Set<String> negativeSet;
	private Set<String> positiveSet;
	@Override
	protected void setup(Context context)
			throws IOException, InterruptedException {
		Configuration conf = context.getConfiguration();
		negativeSet = new HashSet<String>();
		positiveSet = new HashSet<String>();
		for(String negWord : conf.get("negativewords").split(",")) {
			negativeSet.add(negWord);
		}
		for(String posWord : conf.get("positivewords").split(",")) {
			positiveSet.add(posWord);
		}
	}
	@Override
	protected void map(LongWritable key, Text value, Context context)
			throws IOException, InterruptedException {
		double positiveWords = 0;
		double negativeWords = 0;
		double otherWords = 0;
		double result = 0;
		try {
		String[] row = value.toString().split(",");
		String tweet = row[10];
		tempAirline.set(row[5]);
		String[] splitedTweet = tweet.split(" ");
		for(int i = 0; i < splitedTweet.length; i++) {		
			if(splitedTweet[i] != " ") {
				if(negativeSet.contains(splitedTweet[i].trim().replaceAll("[^a-zA-Z0-9]", ""))) {
					negativeWords++;
				}else if(positiveSet.contains(splitedTweet[i].trim().replaceAll("[^a-zA-Z0-9]", ""))) {
					positiveWords++;
				}else {
					otherWords++;
				}
			}
		}
		try {
			result = (positiveWords - negativeWords)/(positiveWords + negativeWords + otherWords);
			logger.info("$$$$$$$$$$$$$$$$$$$$$$$$$$");
			logger.info("RESULT: " +result );
			logger.info("$$$$$$$$$$$$$$$$$$$$$$$$$$");
		}catch(ArithmeticException e) {
			result = 1;
		}
		tempResult.set(result);
		context.write(tempAirline, tempResult);
	}catch(Exception e) {
		logger.error(" ", e);
	}
    }
}

Most interesting part is in my log I am getting the desired output that is:

221.0 Southwest 
312.0 US Airways 
41.0 United 
90.0 Virgin America