One of our clients is running Cloudera Manager Express 5.3.2 and CDH 5.3.2.
There is a strange problem with the Impala Catalog Server.
It has Bad Health in Cloudera Manager which shows the following messages:
StateStore Connectivity Bad
This Catalog Server is not connected to its StateStore.
I can't find anything useful in the log files of the Catalog Server or the StateStore.
Also everything seems to be working properly and the metadata changes are relayed successfully (I performed several tests using 2 different Data Nodes).
In the StateStore Web UI I can see the Catalog Server in the list of the subscribers.
Subscribed topics: 1
Transient entries: 0
If I stop the Catalog Server this entry disappears so it is not a stale one.
I even tried to delete the whole Impala Service and install it again using different node for the StateStore and the Catalog Server. However nothing seems to be helping.
Kerberos and SSL are not used in the cluster.
I suspect that there is only a problem with the Cloudera Manager health check but I have no idea how to troubleshoot further
Please let me know if you have any suggestions.