05-08-2017 07:25 AM
By anychance will there be a possibility in future to restrict hive tables from impala. I am looking for an option to restrict impala tables/partitions to reduce the load in impala-catalog but still allow them queryable from hive.
Thanks and Regards
05-10-2017 09:16 PM
there has not been significant interest in such a feature, so we have not considered it.
It might be worth invstigating the current options to see if they can help:
1. We recommend running the catalogd with the default configuration of --load_catalog_in_background=false. That way the table metadata is only loaded in the catalogd when an Impala query requests it. So, as long as you only access certain tables from Hive and not Impala, the metadata for those tabes will not burden the catalogd.
2. If memory on the catalogd is a problem, you should be able to wipe the cached metadata with "invalidate metadata <table>".