Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

mapred.output.committer.class

Explorer

Could someone confirm the correct practice for setting the mapred.output.committer.class property? I was able to get it to work only if I set the property in my OutputFormat class. I tried

1) setting it in my SerDe class

2) putting it in mapred-site.xml

but in both cases there seems to be no effect. Thank you.

1 ACCEPTED SOLUTION

Super Guru

@Dayou Zhou

can you please set this property as well to make it work

mapreduce.use.directfileoutputcommitter=true

View solution in original post

4 REPLIES 4

Super Guru

@Dayou Zhou

can you please set this property as well to make it work

mapreduce.use.directfileoutputcommitter=true

Explorer

Hi @Sunile Manjee, thanks for the reply. I can try this but my understanding was that mapred.output.committer.class is for old API and mapreduce.use.directfileoutputcommitter is for new API, and it's an either-or. So would it really help to use both?

Super Guru

ok I understand. I have only seen it implemented like this:

committer = ReflectionUtils.newInstance(conf.getClass(
          "mapred.output.committer.class", DirectFileOutputCommitter.class,
          org.apache.hadoop.mapred.OutputCommitter.class), conf);
    }

Super Guru

please advise if this has answered your quesiton