mapred.output.committer.class

[#1]

Could someone confirm the correct practice for setting the mapred.output.committer.class property? I was able to get it to work only if I set the property in my OutputFormat class. I tried

1) setting it in my SerDe class

2) putting it in mapred-site.xml

but in both cases there seems to be no effect. Thank you.

[#2 · Accepted solution]

@Dayou Zhou

can you please set this property as well to make it work

mapreduce.use.directfileoutputcommitter=true

[#3]

Hi @Sunile Manjee, thanks for the reply. I can try this but my understanding was that mapred.output.committer.class is for old API and mapreduce.use.directfileoutputcommitter is for new API, and it's an either-or. So would it really help to use both?

[#4]

ok I understand. I have only seen it implemented like this:

committer = ReflectionUtils.newInstance(conf.getClass(
          "mapred.output.committer.class", DirectFileOutputCommitter.class,
          org.apache.hadoop.mapred.OutputCommitter.class), conf);
    }

[#5]

please advise if this has answered your quesiton