Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

Why does an Impala query fail if DDL is executed when impala statestore is down?

Highlighted

Why does an Impala query fail if DDL is executed when impala statestore is down?

New Contributor

In the official documentation for impala here, the statestore component has a statement:

If you issue a DDL statement while the statestore is down, the queries that access the new object the DDL created will fail.

Correct me if I'm wrong but the metadata changes that are made through Impala SQL are propagated to Impala Daemons from the Impala Catalog Service. Then why would the query on the new object generated through a DDL fail if the statestore is down?

1 REPLY 1

Re: Why does an Impala query fail if DDL is executed when impala statestore is down?

Master Collaborator

The catalog daemon uses the statestore to broadcast metadata changes to all coordinator Impala daemons. If the statestore is down, that broadcast can't happen until it comes back up.

 

This is probably not that relevant immediately, but we've been working on a metadata rearchitecture that reworks this so that the metadata is fetched directly from the catalog and the statestore is only used for invalidating old metadata. (If SYNC_DDL is enabled this is complicated a little further, but I won't go into the gory details).