Ambari Metric Collector: Error sending metric to server. timed out

[#1]

Had a disk full issue. After making some space in /var then trying to restart Metric Collector from Ambari, got error:

----------error ---------------

2015-11-24 11:35:54,281 [INFO] controller.py:110 - Adding event to cache,  : {u'metrics': [], u'collect_every': u'15'}
2015-11-24 11:35:54,281 [INFO] main.py:65 - Starting Server RPC Thread: /usr/lib/python2.6/site-packages/resource_monitoring/main.py start
2015-11-24 11:35:54,281 [INFO] controller.py:57 - Running Controller thread: Thread-1
2015-11-24 11:35:54,282 [INFO] emitter.py:45 - Running Emitter thread: Thread-2
2015-11-24 11:35:54,282 [INFO] emitter.py:65 - Nothing to emit, resume waiting.
2015-11-24 11:36:54,283 [INFO] emitter.py:91 - server: http://xxxxxxx.com:6188/ws/v1/timeline/metrics
2015-11-24 11:37:44,334 [WARNING] emitter.py:74 - Error sending metrics to server. timed out
2015-11-24 11:37:44,334 [WARNING] emitter.py:80 - Retrying after 5 ...

[#2]

@Mike Li

I discussed this with Paul...try this

Stop the Metrics Collector process using Ambari and make sure all ams related processes are also stopped

ps aux | grep ams

if any are still alive

kill -15 <pid>

then restart the Metrics Collector

[#3]

Following Neeraji suggestions, I stopped AMS first, made user no ams processes running, then restart AMS. Metric Monitors (agents) now all started, and the Metrics Collector is still stuck at 35% .... Saw the following output :

[#4]

@Mike Li let's give some more time and see what happens

[#5]

@Mike Li

[#6]

Hi Neeraj im facing this error Metrics Collector - ZooKeeper Server Process :Connection failed: [Errno 111] Connection refused to dcvdevhadnn.eu.scor.local:61181

[#7]

Have you resolved this?

If your cluster is kerberized, AMS in distributed mode, and AMS Collector cannot start, then try setting the AMS RS principal and keytab to AMS HBase master principal and keytab.That's a bug we uncovered recently in Ambari-2.1.2.

For other possible issues check this

https://cwiki.apache.org/confluence/display/AMBARI...

[#8 · Accepted solution]

I just had this issue and this is how it was solved.

I added this to ams-hbase-site :: hbase.zookeeper.property.tickTime = 6000 and then restarted AMS

[#9]

How to restart AMS service

[#10]

where and how can I add this parameter? I am having same issues.

thanks