Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Unable to process alert for an invalid cluster named

avatar
Contributor

i am getting following logs in my ambari-server logs . I check the database and there is no entry of this cluster name "oldclustername" in any tables(i grepped on my dump file and also checked manually in all tables). I named my cluster as "oldclustername" but then renamed it to "newclustername" but logs are still showing these messages. Only table I see suspicious entries is ambari.request which has cluster id=-1 and 2, where cluster_id correctly maps to my newclustername but -1 looks like a wrong entry. Any suggestions where should I look?

23 Jun 2016 23:38:32,233 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert yarn_nodemanager_health for an invalid cluster named oldclustername
23 Jun 2016 23:38:32,361 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named oldclustername
23 Jun 2016 23:38:32,362  WARN [alert-event-bus-2] AlertReceivedListener:248 - Cluster lookup failed for clusterName=oldclustername
23 Jun 2016 23:38:32,362  WARN [alert-event-bus-2] AlertReceivedListener:134 - Received an alert for ambari_agent_disk_usage which is a definition that does not exist anymore
23 Jun 2016 23:38:32,362 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert kafka_broker_process for an invalid cluster named oldclustername
23 Jun 2016 23:38:32,942 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert datanode_webui for an invalid cluster named oldclustername
23 Jun 2016 23:38:32,943 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert datanode_process for an invalid cluster named oldclustername
23 Jun 2016 23:38:32,943  WARN [alert-event-bus-2] AlertReceivedListener:248 - Cluster lookup failed for clusterName=oldclustername
23 Jun 2016 23:38:32,943  WARN [alert-event-bus-2] AlertReceivedListener:134 - Received an alert for ambari_agent_disk_usage which is a definition that does not exist anymore
23 Jun 2016 23:38:32,943 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert yarn_nodemanager_webui for an invalid cluster named oldclustername
23 Jun 2016 23:38:32,943 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named oldclustername
23 Jun 2016 23:38:32,943 ERROR [alert-event-bus-2] AlertReceivedListener:329 - Unable to process alert yarn_nodemanager_health for an invalid cluster named oldclustername
1 ACCEPTED SOLUTION

avatar
Super Collaborator

There is a bug on older Ambari versions where if you rename a cluster (or delete a cluster), you'll get these errors in your logs. It's because the agents have not un-scheduled the old cluster's alerts. You can just restart the agents to have these errors go away.

View solution in original post

3 REPLIES 3

avatar
Super Collaborator

There is a bug on older Ambari versions where if you rename a cluster (or delete a cluster), you'll get these errors in your logs. It's because the agents have not un-scheduled the old cluster's alerts. You can just restart the agents to have these errors go away.

avatar
Contributor

I restarted the agents but looks like the errors are still present. I see the old cluster name in a execution_commands table. I have identified the task_id which are having the command haivng my old clustername. Should I delete them from the database?

avatar
Contributor

Did you solved the issue? We have the same erros in our 2.4.2 cluster.

ERROR [alert-event-bus-2] AlertReceivedListener:341 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named XXXX.

WARN [qtp-ambari-client-68] ViewRegistry:855 - Could not find the cluster identified by XXXX.