Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

When querying a VIEW, query planning takes a long time

Solved Go to solution

Re: When querying a VIEW, query planning takes a long time

Contributor

@Lars Volker

 

I have a question for you.

How long the metadata loaded from Hive metastore by Impala Catalog Daemon stay in memory?

 

I'm using Impala 2.7 ( KUDU ).

It seems the metadata is flushed more often than before.

 

Is there any configuration for life cycle for metadata in catalog daemon has?

 

@gaurang

I'm asking this question here because I guess @Lars Volker answer can help resolve your issue.

 

Thank you

Gatsby

Re: When querying a VIEW, query planning takes a long time

Expert Contributor

@Gatsby - I don't know for sure, but I don't think metadata is flushed periodically. There also don't seem to be any configuration options of catalogd around metadata caching. Instead, the catalog should flush metadata when requested by "invalidate metadata" or by "refresh" or when a DDL statement makes changes to a table's metadata. Such changes should show up in the logfiles however.

Re: When querying a VIEW, query planning takes a long time

Contributor
yeap. you're right. I will take a look log.

Thank you
Gatsby
Highlighted

Re: When querying a VIEW, query planning takes a long time

Contributor

@gaurang

 

Today, I had some issue with slow quries.

And, the issue was related to metadata Catalog Daemon caches.

 

 

How often do you make quries to that TABLE/VIEW ( I don't think your issue is related to VIEW )?

 

In my case, metadata for TABLE was reloaded very often because Catalog Daemon flushes out metadata.

 

Take a look your catalog daemon and check if TABLE metadata is cached.

 

Gatsby

Re: When querying a VIEW, query planning takes a long time

Master Collaborator

@gaurang would you be open to sharing your CREATE TABLEs, CREATE VIEW and the query that has slow planning time? No need for the data, just that should be sufficient for us to understand better what's going on.

 

Like Lars said, you are probably hitting IMPALA-4242 which explains the slow equivalence class computation, but I'd also like to understand the slow single-node planning time.

 

Thanks!