https://community.cloudera.com/t5/Support-Questions/Hive-external-table-partition-update-automatically/m-p/313890#M225787 <P>The "msck repair table ..." command does not really read new data files, but adds new partitions (subdirectories in HDFS) in table metadata.</P><P>What you could do is to create in advance all the partitions (for month or more) - initially empty- and run the "repair" command just once:</P><P>hdfs dfs -mkdir /user/data/year=2021/month=04/day=1</P><P>...</P><P>hdfs dfs -mkdir /user/data/year=2021/month=04/day=30</P><P>hive&gt;msck repair table &lt;table_name&gt;</P><P>&nbsp;</P><P>When You put your log files inside one of these directories, they will be immediately visible from Hive (just set the correct permissions using Ranger or hdfs).</P><P>&nbsp;</P><P>Maybe You can repeat this operations (create directories and "repair table") during logs maintenance, as you should have some policies to remove old logs</P><P>&nbsp;</P><P>Hope this helps</P> Wed, 31 Mar 2021 14:59:39 GMT giovannimori 2021-03-31T14:59:39Z