Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Why does an Impala query fail if DDL is executed when impala statestore is down?

Why does an Impala query fail if DDL is executed when impala statestore is down?

New Contributor

In the official documentation for impala here, the statestore component has a statement:

If you issue a DDL statement while the statestore is down, the queries that access the new object the DDL created will fail.

Correct me if I'm wrong but the metadata changes that are made through Impala SQL are propagated to Impala Daemons from the Impala Catalog Service. Then why would the query on the new object generated through a DDL fail if the statestore is down?

1 REPLY 1
Highlighted

Re: Why does an Impala query fail if DDL is executed when impala statestore is down?

Master Collaborator

The catalog daemon uses the statestore to broadcast metadata changes to all coordinator Impala daemons. If the statestore is down, that broadcast can't happen until it comes back up.

 

This is probably not that relevant immediately, but we've been working on a metadata rearchitecture that reworks this so that the metadata is fetched directly from the catalog and the statestore is only used for invalidating old metadata. (If SYNC_DDL is enabled this is complicated a little further, but I won't go into the gory details).

Don't have an account?
Coming from Hortonworks? Activate your account here