Skip to main content
chriswalton007
chriswalton007
Gold ⭐️
March 5, 2019
Solved

HUE Clear Cache, Perform Incremental Metadata Update, Invalidate all Metadata and rebuild index

Hi,

I'm trying to understand a little more about what these options do behind the scenes when pressed?  I found this post, but it never got any responses (http://community.cloudera.com/t5/Web-UI-Hue-Beeswax/What-is-under-the-hood-of-Hue-Impala-editor-Clear-cache/m-p/59814#M2432)

Clear Cache
This is the one I am not too sure what happens

Perform Incremental metadata Update
I assume this issues a refresh command for all tables within the current database which is been viewed?  If no database is veiwed does it do it for everything?

Invalidate All metadata and rebuild index
Same as above, but rather than refresh it invalidates full metadata?  If no database is veiwed does it do it for everything?

Harsh J
Harsh JAnswer
Gold ⭐️⭐️
March 6, 2019
> Clear Cache
> This is the one I am not too sure what happens

It appears to clear the cached entries within Hue frontend, so the metadata for assist and views is loaded again from its source (Impala, etc.). I don't see it calling a refresh on the tables, but it is possible I missed some implicit action.

> Perform Incremental metadata Update
> I assume this issues a refresh command for all tables within the current database which is been viewed? If no database is veiwed does it do it for everything?

This will compare HMS listing against Impala's for the DB in context and run specific "INVALIDATE METADATA [[db][.table]];" for the missing ones in Impala.

Yes, if no DB is in the context, it will equate to running a global "INVALIDATE METADATA;"

> Invalidate All metadata and rebuild index

This runs a plain "INVALIDATE METADATA;"
View in thread

2 replies

Harsh J
Gold ⭐️⭐️
Harsh JAnswer
Gold ⭐️⭐️
March 6, 2019
> Clear Cache
> This is the one I am not too sure what happens

It appears to clear the cached entries within Hue frontend, so the metadata for assist and views is loaded again from its source (Impala, etc.). I don't see it calling a refresh on the tables, but it is possible I missed some implicit action.

> Perform Incremental metadata Update
> I assume this issues a refresh command for all tables within the current database which is been viewed? If no database is veiwed does it do it for everything?

This will compare HMS listing against Impala's for the DB in context and run specific "INVALIDATE METADATA [[db][.table]];" for the missing ones in Impala.

Yes, if no DB is in the context, it will equate to running a global "INVALIDATE METADATA;"

> Invalidate All metadata and rebuild index

This runs a plain "INVALIDATE METADATA;"
Translate
    Krishna
    Krishna
    Silver ⭐️
    September 25, 2019

    Hi Harsha, 

     

    Thanks for the explanation. In extension to the topic, I need small clarification - we recently implemented sentry on impala, based on below KB [1] article, we can't execute "Invalidate all metadata and rebuild index" and "Perform incremental metadata update" , since we don't have access to all the DB's, it's fair as well. Now my question is -

    1. I am not able to see new DB in Hue impala, I can see the same from beeline or impala shell. How to fix or solve this ?

    2. I can execute invalidate metadata on table from impala shell but I have 50+ DB's and 10's of tables in each db. Is there any option to run invalidate metadata ion DB level instead of individual table?

    [1] https://my.cloudera.com/knowledge/INVALIDATE-METADATA--Sentry-Enabled--ERROR?

    id=71141

     

    Thanks

    Krishna

     

    Translate
    Terms & ConditionsAccessibility statement