Metrics Collector doesn't start in HDP 3.1

[#1]

Hi All,

Can somebody let me know why Ambari Metrics Collector is not starting every time I start it fails.

Log file says:

2019-08-20 09:13:25,420 INFO [agent-report-processor-1] ServiceComponentHostImpl:1054 - Host role transitioned to a new state, serviceComponentName=METRICS_COLLECTOR, hostName=gspdhd04.gsp.local, oldState=STARTING, currentState=STARTED
2019-08-20 09:13:28,379 ERROR [ambari-client-thread-116] MetricsRequestHelper:112 - Error getting timeline metrics : Connection refused (Connection refused)
2019-08-20 09:13:28,379 ERROR [ambari-client-thread-116] MetricsRequestHelper:119 - Cannot connect to collector: SocketTimeoutException for gspdhd04.gsp.local
2019-08-20 09:13:30,370 ERROR [ambari-client-thread-115] MetricsRequestHelper:112 - Error getting timeline metrics : Connection refused (Connection refused)
2019-08-20 09:13:30,370 ERROR [ambari-client-thread-115] MetricsRequestHelper:119 - Cannot connect to collector: SocketTimeoutException for gspdhd04.gsp.local
2019-08-20 09:13:44,388 ERROR [ambari-client-thread-34] MetricsRequestHelper:112 - Error getting timeline metrics : Connection refused (Connection refused)
2019-08-20 09:13:44,388 ERROR [ambari-client-thread-34] MetricsRequestHelper:119 - Cannot connect to collector: SocketTimeoutException for gspdhd04.gsp.local
2019-08-20 09:13:45,373 ERROR [ambari-client-thread-162] MetricsRequestHelper:112 - Error getting timeline metrics : Connection refused (Connection refused)
2019-08-20 09:13:45,374 ERROR [ambari-client-thread-162] MetricsRequestHelper:119 - Cannot connect to collector: SocketTimeoutException for gspdhd04.gsp.local
2019-08-20 09:14:00,384 ERROR [ambari-client-thread-162] MetricsRequestHelper:112 - Error getting timeline metrics : Connection refused (Connection refused)
2019-08-20 09:14:00,385 ERROR [ambari-client-thread-162] MetricsRequestHelper:119 - Cannot connect to collector: SocketTimeoutException for gspdhd04.gsp.local
2019-08-20 09:14:00,387 ERROR [ambari-client-thread-35] MetricsRequestHelper:112 - Error getting timeline metrics : Connection refused (Connection refused)
2019-08-20 09:14:00,387 ERROR [ambari-client-thread-35] MetricsRequestHelper:119 - Cannot connect to collector: SocketTimeoutException for gspdhd04.gsp.local
2019-08-20 09:14:07,703 INFO [agent-report-processor-1] HeartbeatProcessor:647 - State of service component METRICS_COLLECTOR of service AMBARI_METRICS of cluster 2 has changed from STARTED to INSTALLED at host gspdhd04.gsp.local according to STATUS_COMMAND report
2019-08-20 09:14:15,370 INFO [ambari-client-thread-162] AMSPropertyProvider:626 - METRICS_COLLECTOR host is not live. Skip populating resources with metrics, next message will be logged after 1000 attempts.

I am not able to understand this.

[#2]

@Manjunath - Can you check if your metrics collector is tuned as per the below document:

https://cwiki.apache.org/confluence/display/AMBARI/Configurations+-+Tuning

[#3]

Hi ngarg,

The document is really handy but for me the issue has been resolved after moving the ambari metrics to different node initially it was in datanode now moved to Namenode and I am still finding the reason how it works here.

Regards,

Manjunath P N

[#4]

Hi,

The document is really handy but for me, the issue has been resolved after moving the Ambari metrics to different node initially it was in data node now moved to Namenode and I am still finding the reason how it works here.

Regards,

Manjunath P N