https://community.cloudera.com/t5/Support-Questions/How-to-overwrite-an-existing-output-file-dir-during/m-p/214810#M176722 <P>Below two steps to delete the output directory(not recommended) in <STRONG><A href="http://data-flair.training/blogs/hadoop-mapreduce-introduction-tutorial-comprehensive-guide/">MapReduce</A></STRONG>:<BR /><STRONG>1) using shell:</STRONG><BR /><CODE>bin/hadoop dfs -rmr /path/to/your/output/</CODE><BR /><STRONG>2) JAVA API:</STRONG></P><PRE><CODE>// configuration should contain reference to your namenode FileSystem fs = FileSystem.get(new Configuration()); // true stands for recursively deleting the folder you gave fs.delete(new Path(”/path/to/your/output”), true);</CODE></PRE><P>If you want to override the existing:<BR />Need to overwrite the <STRONG><A href="http://data-flair.training/blogs/outputformat-recordwriter-hadoop-mapreduce/">Hadoop OutputFormat</A></STRONG> class:</P><PRE><CODE>public class OverwriteOutputDirOutputFile extends TextOutputFormat{ @Override public void checkOutputSpecs(FileSystem ignored, JobConf job) throws FileAlreadyExistsException, InvalidJobConfException, IOException { // Ensure that the output directory is set and not already there Path outDir = getOutputPath(job); if (outDir == null &amp;&amp; job.getNumReduceTasks() != 0) { throw new InvalidJobConfException(”Output directory not set in JobConf.”); } if (outDir != null) { FileSystem fs = outDir.getFileSystem(job); // normalize the output directory outDir = fs.makeQualified(outDir); setOutputPath(job, outDir); // get delegation token for the outDir’s file system TokenCache.obtainTokensForNamenodes(job.getCredentials(), new Path[] {outDir}, job); // check its existence /* if (fs.exists(outDir)) { throw new FileAlreadyExistsException(”Output directory ” + outDir + ” already exists”); }*/ } } </CODE></PRE><P>}</P> Sat, 07 Oct 2017 17:17:44 GMT shreyag1207 2017-10-07T17:17:44Z