are the statistics up to date on the tables? It should be able to do a semi join reduction. Updating stats may help: https://cwiki.apache.org/confluence/display/Hive/Column%2BStatistics%2Bin%2BHive ... View more

From the directory listing, your table must have "transactional=true" property, i.e. it's an ACID table. That means that Insert Overwrite will create a base_x directory where it will put the result of the insert (new data) there. Any data that existed before, will remain in the table but will not be visible to readers that start after Insert Overwrite finished. Old data will be physically removed once Compaction runs over this table/partition. ... View more

The contents of a managed table should be manage by Hive, you cannot just write files there. To do that you have to use External table. To add data to a managed table you should use Insert or Load Data or some other Hive command, but not just drop files there. ... View more

Is this a managed or external table? If managed, do you have any files in the table that don't match 0000_0, 0000_0_copy_1 or bucket_0 pattern? ... View more

Have you considered SQL Merge statement? ... View more

This may work beeline> !run ///C:/ScriptFile ... View more

Hive metastore is a service. That may run in embedded mode and in stand alone mode. You can have several instances running in your cluster, all instances must share the same backend RDMBS. This should help: https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-RemoteMetastoreServer ... View more

Do you have the standalone metastore running? that is where compaction jobs are actually generated and submitted. ... View more

desc formatted <table> <column> https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-Examples ... View more