Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive : merging small data files into a big file

Hive : merging small data files into a big file

Explorer

We had a Hive table which has its "location" in a hdfs directory. The location directory has a bunch of small data files which represents the data of the table. Data keeps coming into the location directory and so numerous small files are created all the time.

But for performance we want to merge all these small files into a larger file on a periodic basis.

Whats the best way to do this? I hear that Hive itself has a merge option?

Appreciate the insights.