It was as simple as removing the /.snapshot from the path?! Thanks @Jitendra Yadav , clearly time for me to take a break...! ... View more

@R Wys There is no problem with hive here, hive has generated an execution plan with no reduce phase in your case. you can see the plan by running 'explain select*from myTable where daily_date='2015-12-29' limit 10' ... View more

No problem, we all do these mistakes at times 🙂 ... View more

Thanks Benjamin, great information. About option a) - in this case a new (and query-able) meta-column called daily_date in the nice format would be created in the final table, wouldn't it? [Edit:just done it, yes it is] To make this work as an automated process where hive -e is called in a shell script, I would just need to set the new daily_date as a variable somewhere before the hive call (I think). Then: INSERT OVERWRITE TABLE final PARTITION (daily_date=${nice_date}) SELECT facts, otherFact<exclude daily_date> FROM staging ; Should work! Thanks again. ... View more

Any solutions that work long term? ... View more