Support Questions
Find answers, ask questions, and share your expertise

Re-install ambari-metrics override group_config in storm and kafka

Highlighted

Re-install ambari-metrics override group_config in storm and kafka

Explorer

Hi,

We found an issue after re-installing ambari-metrics in the HDP cluster.

We first install ambari-metrics in 50 hosts in a new cluster, then install kafka in 20 hosts and storm in the other 30. Each kafka broker has its own group_config to override the broker.id, broker.rack and host.name. Same with storm where we overridden for each supervisor and nimbus the storm.local.hostname property.

Everythings were good at that point.

We faced some issue with ambari-metrics and failed to solve them (that's another story). We decide to re-install the component.

We proceed like that:

  • Remove znode /ams-hbase-unsecured from the embedded zookeeper (localhost:21181)
  • Stop all ambari-metrics instances
  • Delete ambari-metrics services from ambari
  • Remove all rpm related to ambari-metrics
  • Remove all folder related to ambari-metrics and hbase(/var/lib/ambari-metrics-collector, ...)
  • Re-install ambari-metrics service from ambari.

After re-installing with success, we discover that each group_config for kafka and storm were overridden with all the same value. All had the broker.id set to 1, broker.rack = rack1 and host.name= myhostname1.mycompany.com. Same for storm.

We test once more the procedure in another cluster with the same configuration and it's happened exactly the same.

Our cluster:

  • HDP 2.5.0.0-1245 with kafka 0.10, Storm 1.0
  • Ambari 2.4.0.1-1

Did someone faced the same issue?

Can someone confirm that is a bug in ambari?

Thanks