Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What / When flush out table metadata in catalog daemon?

SOLVED Go to solution

What / When flush out table metadata in catalog daemon?

Contributor

Hello,

 

I have a question about metadata loaded in Impala catalog daemon.

 

If I understand correctly, catalog daemon read TABLE metadata from Hive metadata store and caches in memory.

My question is what trigger flushing out this cached TABLE metadata.

 

The reason I ask this question is that I noticed that TABLE metadata is flushed out from catalog daemon after some time. And, since TABLE metadata doesn't exist, catalog daemon tries to load metadata again from metastore.

 

Is there some kind of configuration I can set to control TABLE metadata lifecycle ( or memory )?

 

Thank you

Gatsby

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: What / When flush out table metadata in catalog daemon?

Expert Contributor

Quoting my reply in this thread:

 

I don't know for sure, but I don't think metadata is flushed periodically. There also don't seem to be any configuration options of catalogd around metadata caching. Instead, the catalog should flush metadata when requested by "invalidate metadata" or by "refresh" or when a DDL statement makes changes to a table's metadata. Such changes should show up in the logfiles however.

3 REPLIES 3

Re: What / When flush out table metadata in catalog daemon?

Expert Contributor

Quoting my reply in this thread:

 

I don't know for sure, but I don't think metadata is flushed periodically. There also don't seem to be any configuration options of catalogd around metadata caching. Instead, the catalog should flush metadata when requested by "invalidate metadata" or by "refresh" or when a DDL statement makes changes to a table's metadata. Such changes should show up in the logfiles however.

Highlighted

Re: What / When flush out table metadata in catalog daemon?

Contributor

From log history, I found out someone keep running 'invalidate metadata' without table name. :(

 

Thank you for your comment

Gatsby

Re: What / When flush out table metadata in catalog daemon?

Expert Contributor

Sure thing, I'm glad you found the culprit. :)