https://community.cloudera.com/t5/Support-Questions/merge-file-in-hdfs/m-p/216281#M178190 <P><A rel="user" href="https://community.cloudera.com/users/45384/evaloschin.html" nodeid="45384">@eric valoschin</A> </P><P>Can you try the following command </P><PRE>hadoop jar /usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-streaming-2.7.3.2.5.3.0-37.jar \ -Dmapred.reduce.tasks=1 \ -input "&lt;path-to-input-directory&gt;" \ -output "&lt;path-to-output-directory&gt;" \ -mapper cat \ -reducer cat</PRE><P>make sure which version of <STRONG>hadoop streaming jar </STRONG>you are using by going to </P><PRE>/usr/hdp</PRE><P>then give the <STRONG>input path</STRONG> and make sure the <STRONG>output directory</STRONG> is <STRONG>not existed</STRONG> as this job will <STRONG>merge the files</STRONG> and creates the <STRONG>output directory</STRONG> for you.</P><P><STRONG>Here what i tried:-</STRONG></P><PRE>#hdfs dfs -ls /user/yashu/folder2/ Found 2 items -rw-r--r-- 3 hdfs hdfs 150 2017-09-26 17:55 /user/yashu/folder2/part1.txt -rw-r--r-- 3 hdfs hdfs 20 2017-09-27 09:07 /user/yashu/folder2/part1_sed.txt</PRE><PRE>#hadoop jar /usr/hdp/2.5.3.0-37/hadoop-mapreduce/hadoop-streaming-2.7.3.2.5.3.0-37.jar \ &gt; -Dmapred.reduce.tasks=1 \ &gt; -input "/user/yashu/folder2/" \ &gt; -output "/user/yashu/folder1/" \ &gt; -mapper cat \ &gt; -reducer cat</PRE><P>Folder2 having <STRONG>2 files</STRONG> after running the above command, i am storing the merged files to <STRONG>folder1</STRONG> directory and the<STRONG> 2</STRONG> files got merged into <STRONG>1 file</STRONG> as you can see below. </P><PRE>#hdfs dfs -ls /user/yashu/folder1/ Found 2 items -rw-r--r-- 3 hdfs hdfs 0 2017-10-09 16:00 /user/yashu/folder1/_SUCCESS -rw-r--r-- 3 hdfs hdfs 174 2017-10-09 16:00 /user/yashu/folder1/part-00000</PRE> Tue, 10 Oct 2017 03:07:17 GMT Shu_ashu 2017-10-10T03:07:17Z