This Impala daemon is not ready to accept user requests

[#1]

Hello Experts,

I am getting error as below when trying to INVALIDATE METADATA command, as my Hive tables are not reflecting in Impala. Could you please help

[#2 · Accepted solution]

To solve the issue

Go to Services Configuration

Then select Impala State Service, Impala Catlog service and Impala server and start all these services.

Once these services are up and running you can give command INVALIDATE METADATA in impala query editor after which all the tables and DB in hive will be reflected in Impala

[#3]

Hi Could you explain why a INVALIDATE METADATA throws such an error?  Are there any circumvent options in terms of coding and to avoid impala service restarts?

[#4]

@Mara the previous solution is a bit out of date. We fixed this in CDH 5.14 and up so that clients can't connect until the service is ready. So that would avoid the issue.

The issue happened in older versions during the impala daemon startup. It can happen for a longer period when some of the services for the impala cluster (catalog or statestore) are not operational, because the impala daemon can't finish startup in those cases.

[#5]

Hi Thanks for your response.  Could you explain more about catalog or statestore not being operational and what causes them?  We are on CDH 5.11 and how do we identify this situation in our cluster?

[#6]

Ahh 5.11, there's been so many Impala improvements since then!

This happens when the Impala daemon can't load the initial catalog (i.e. database and table metadata). The catalog and statestore roles are both involved in the catalog loading, so if the impala daemon isn't able to communicate with those roles, or those are not started or healthy then that could lead to these symptoms. You should be able to see in Cloudera Manager if they're started and if there are any warnings or errors being flagged.

It might also be just that the catalog is slow to load (maybe there's a lot of metadata or something else is unhealthy). You would need to look at the logs of the impala daemon you're connecting and maybe the catalog to see what it's doing and why its slow.

I know this doesn't address your immediate problem, but we've seen a lot of these metadata/catalog problem go away with later versions - CDH5.16 or CDH6+, and particularly by moving to a dedicated coordinator/executor topology - https://docs.cloudera.com/documentation/enterprise/5-16-x/topics/impala_dedicated_coordinator.html.