question Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors in Support Questions

After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

cspencer — Thu, 01 Oct 2015 03:27:19 GMT

From ambari-server 2 log:

28 Sep 2015 19:04:23,498 ERROR [qtp-client-6858] MetricsPropertyProvider:183 - Error getting timeline metrics. Can not connect to collector, socket error. 
28 Sep 2015 19:04:28,989 INFO [AlertNoticeDispatchService] AlertNoticeDispatchService:279 - There are 1 pending alert notices about to be dispatched... 
28 Sep 2015 19:04:28,993 INFO [alert-dispatch-96] EmailDispatcher:88 - Sending email: org.apache.ambari.server.state.alert.AlertNotification@35b0f886 
28 Sep 2015 19:04:29,245 WARN [ambari-hearbeat-monitor] HeartbeatMonitor:154 - Heartbeat lost from host blpd813-priv.bhdc.att.com 
28 Sep 2015 19:04:33,519 ERROR [qtp-client-6858] MetricsReportPropertyProvider:223 - Error getting timeline metrics. Can not connect to collector, socket error.

From the ambari-alerts log:

2015-09-28 18:34:01,981 [CRITICAL] [YARN] [yarn_nodemanager_webui] (NodeManager Web UI) Connection failed to http://blpd852-priv.bhdc.att.com:8042 (<urlopen error timed out>) 
2015-09-28 18:34:03,088 [CRITICAL] [YARN] [yarn_nodemanager_webui] (NodeManager Web UI) Connection failed to http://blpd820-priv.bhdc.att.com:8042 (<urlopen error timed out>) 
2015-09-28 18:34:52,711 [OK] [YARN] [yarn_nodemanager_webui] (NodeManager Web UI) HTTP 200 response in 0.000s 
2015-09-28 18:34:53,709 [OK] [YARN] [yarn_nodemanager_webui] (NodeManager Web UI) HTTP 200 response in 0.000s 
2015-09-28 19:04:05,354 [CRITICAL] [YARN] [yarn_nodemanager_webui] (NodeManager Web UI) Connection failed to http://blpd810-priv.bhdc.att.com:8042 (<urlopen error timed out>) 
2015-09-28 19:04:56,024 [OK] [YARN] [yarn_nodemanager_webui] (NodeManager Web UI) HTTP 200 response in 0.000s

Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

abajwa — Thu, 01 Oct 2015 03:46:33 GMT

Is Ambari Metrics service up?

Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

pardeep_kumar — Thu, 01 Oct 2015 03:57:43 GMT

Is AMS configured in embedded or distributed mode?

Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

cspencer — Thu, 01 Oct 2015 04:02:00 GMT

I don't know if it is setup in distributed or embedded mode, but I will check on that and if metrics services are up and update you.

Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

cspencer — Thu, 01 Oct 2015 21:43:35 GMT

they are running in embedded mode and ams is up and running

Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

afernandez — Fri, 02 Oct 2015 00:38:31 GMT

Can you create an EAR for this an attach the Ambari Metrics Collector logs?

Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

sidwagle — Fri, 02 Oct 2015 00:46:17 GMT

Instructions on list of items to upload:

https://cwiki.apache.org/confluence/display/AMBARI/Troubleshooting+Guide

Re: After upgraded Ambari from 1.7 to 2.1.1 then started receiving timeline metrics cannot connect collector socket errors

cspencer — Fri, 02 Oct 2015 00:57:41 GMT

https://hortonworks.jira.com/browse/EAR-2354 was created for this issue