Impala Catalogd high availability

[#1]

Hello Cloudera Community,

We have a use case where we cannot afford the restart of catalogd, as the reload of metadata takes many hours and queries are impacted during this long reload.

Is it possible to have multiple catalogd instances with an active / passive setup? In this case the active catalogd would act as the main catalogd and in case of failure, one of the passive catalogds would take the lead becoming active. This would eliminate the huge metadata reload time in case of failure in the catalogd node. 

It would be really interesting to have the feedback of the community about such setup, and see if someone has already tested it.

Thanks a lot for your replies

[#2 · Accepted solution]

Greetings @melmoumni 

Thanks for using Cloudera Community. For now, No to HA for Catalogd. This is coming from the fact stated in [1], wherein the Catalogd Unavailability don't leads to DataLoss & they can be removed to be added on a New Host without any impact. Similarly, the Upstream JIRA [2] remains Unresolved. 

Again, I shall keep the Post available for others to provide their feedback, if they have tested the same. In [2], One User share their experience, which doesn't appear to be Successful. 

Regards, Smarak

[1] https://impala.apache.org/docs/build/html/topics/impala_proxy.html

[2] https://issues.apache.org/jira/browse/IMPALA-2702