Could someone confirm the correct practice for setting the mapred.output.committer.class property? I was able to get it to work only if I set the property in my OutputFormat class. I tried
1) setting it in my SerDe class
2) putting it in mapred-site.xml
but in both cases there seems to be no effect. Thank you.
Hi @Sunile Manjee, thanks for the reply. I can try this but my understanding was that mapred.output.committer.class is for old API and mapreduce.use.directfileoutputcommitter is for new API, and it's an either-or. So would it really help to use both?
ok I understand. I have only seen it implemented like this:
committer = ReflectionUtils.newInstance(conf.getClass( "mapred.output.committer.class", DirectFileOutputCommitter.class, org.apache.hadoop.mapred.OutputCommitter.class), conf); }