Support Questions

Find answers, ask questions, and share your expertise

Problem with Ambari Metrics Collector while upgrading Ambari 2.2.2 ->



My config: HDP on Ubuntu14

I was trying to upgrade Ambari from 2.2.2 to and I followed instructions in

Everything worked fine until I got to the point of Upgrading Ambari Metrics:

apt-get install ambari-metrics-collector

and got error that this package couldn't be found. My /etc/apt/sources.list.d/ambari.list is as follows:

deb Ambari main

All other ambari packages seem to be installed fine (dpkg -l ambari-*):

ii  ambari-agent           amd64                     Ambari Agent
ii  ambari-metrics-assembly      amd64                     Ambari Metrics Assembly
ii  ambari-server          amd64                     Ambari Server

What should I do to install ambari-metrics-collector?


Super Mentor

@Darek Rutkowski

Did you try running the "apt-get install ambari-metrics-collector" again? It might be a temporary n/w issue?

Else you might want to try downloading the ambari offline repository that has the required RPMs from:


For other OS versions offline repositories you can take a look at:


Thanks for the hints.

I did try "apt-get install ambari-metrics-collector" couple of times for a few days with the same result.

I downloaded the ambari offline repository you mentioned, put it to ambari.list file and apt gives me the same answer. If you look into the contents of ambari-\ubuntu14\dists\Ambari\main\binary-amd64\Packages file, there is no such package defined as ambari-metrics-collector.

Maybe all necessary files of ambari-metrics-collector are already installed with different package?

There is /usr/sbin/ambari-metrics-collector file in the system, but I couldn't find the way to check its version.

Super Mentor

@Darek Rutkowski

I did some more research and found that for Ubuntu you do not need to do the following step.

apt-get install ambari-metrics-collector


(For Ubuntu) Just 'apt-get install ambari-metrics-assembly' should be sufficient for upgrading Ambari metrics on Ubuntu 14.

Can you confirm by checking the ambari-metrics-*.jar versions in /usr/lib/ambari-metrics-collector.

# ls /usr/lib/ambari-metrics-collector/ambari-metrics-* 



Yes, these two jars with this version are there.

As I wrote a few minutes ago, I got back to Ambari 2.2.2 and will do the upgrade in a couple of weeks. I have one server with Ubuntu 16 and Ambari 2.5.0 for testing. So, what you wrote is true for Ubuntu 16 too.

Super Mentor


@Darek Rutkowski

Run on of these 2 commands to clear the cache

sudo apt-get clean 
sudo apt-get autoclean

Then rerun you command your

sudo apt-get install ambari-metrics-collector 

The installation should work


Thanks, I tried it and with no success.

As I mentioned in comment to Jay SenSharma -I couldn't find this package in the tarball version of ambari repository, so maybe all necessary files are installed with different one/ones?


@Darek Rutkowski

in your initial steps did you execute the following in sequence

On cluster node running a Metrics Monitor

apt-get clean all
apt-get update
apt-get install ambari-metrics-assembly 

Execute the following command on all hosts running the Metrics Collector

apt-get clean all
apt-get update
apt-get install ambari-metrics-collector 

On the host running the Grafana component

apt-get clean all
apt-get update
apt-get install ambari-metrics-grafana 

Start Ambari Metrics Service.

From Ambari Web, browse to Services > Ambari Metrics select Service Actions then choose Start.

Please revert


Yes, I went strictly with instructions and when I look into bash_history I'm sure I followed this sequence. But I didn't get to the Grafana Component because of error with ambari-metrics-collector.

Start action in Ambari Web is disabled for Ambari Metrics now, which is quite obvious.

I consider preparing new virtual machine for AMS to workaround this problem.


I prepared fresh new virtual machine with Ubuntu16 with /etc/apt/sources.list.d/ambari.list:

deb Ambari main

After installing packages: ambari-agent and ambari-metrics-assembly I noticed that there are the following commands available:

ambari-metrics-collector ambari-metrics-grafana ambari-metrics-monitor

Does it mean that ambari-metrics-collector is already installed and I don't need to install it with separate package?

The same situation is on the server I try to upgrade. How can I verify that ambari-metrics-[collector,grafana,monitor] are correctly upgraded?


@Darek Rutkowski

Its good alwasy to follow the hortonworks documentation.

Can you remove the offending package through Ambari UI?

List of the applications installed add a grep to filter:

sudo apt --installed list | more 

Then remove a program run the following command:

sudo apt-get remove 

To completely remove run the following command:

sudo apt-get remove --purge 

Then try re-installing


The situation gets more complicated 🙂

Now I have problem with ambari-agents which can't connect to ambari-server.

Ambari-server seems to work fine except AMS. Upgraded ambari-agent starts with the following error:

INFO 2017-05-19 12:49:41,405 - loglevel=logging.INFO
INFO 2017-05-19 12:49:41,405 - loglevel=logging.INFO
INFO 2017-05-19 12:49:41,405 - loglevel=logging.INFO
INFO 2017-05-19 12:49:41,406 - Data cleanup thread started
INFO 2017-05-19 12:49:41,408 - Data cleanup started
INFO 2017-05-19 12:49:41,409 - Data cleanup finished
INFO 2017-05-19 12:49:41,437 - Ping port listener started on port: 8670
INFO 2017-05-19 12:49:41,438 - Connecting to Ambari server at https://hadoop-metrics.pip:8440 (
INFO 2017-05-19 12:49:41,438 - Connecting to https://hadoop-metrics.pip:8440
INFO 2017-05-19 12:49:41,623 - Connected to Ambari server hadoop-metrics.pip
INFO 2017-05-19 12:49:41,624 - Started thread pool with 3 core threads and 20 maximum threads
WARNING 2017-05-19 12:49:41,624 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2017-05-19 12:49:41,624 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7fbf67547c90>; currently running: False
ERROR 2017-05-19 12:49:41,629 - Controller thread failed with exception:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/", line 482, in run
    self.actionQueue = ActionQueue(self.config, controller=self)
  File "/usr/lib/python2.6/site-packages/ambari_agent/", line 79, in __init__
    self.statusCommandResultQueue = multiprocessing.Queue() # this queue is filled by StatuCommandsExecutor.
  File "/usr/lib/python2.7/multiprocessing/", line 218, in Queue
    return Queue(maxsize)
  File "/usr/lib/python2.7/multiprocessing/", line 63, in __init__
    self._rlock = Lock()
  File "/usr/lib/python2.7/multiprocessing/", line 147, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1)
  File "/usr/lib/python2.7/multiprocessing/", line 75, in __init__
    sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 30] Read-only file system
ERROR 2017-05-19 12:49:43,628 - Exiting with exception:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/", line 471, in <module>
  File "/usr/lib/python2.6/site-packages/ambari_agent/", line 450, in main
    run_threads(server_hostname, heartbeat_stop_callback)
  File "/usr/lib/python2.6/site-packages/ambari_agent/", line 340, in run_threads
    controller.get_status_commands_executor().kill("AGENT_STOPPED", can_relaunch=False)
AttributeError: 'NoneType' object has no attribute 'kill'
INFO 2017-05-19 12:49:43,630 - Performing cleanup before exiting...
INFO 2017-05-19 12:49:43,630 - Shutting down thread pool
INFO 2017-05-19 12:49:43,630 - Scheduler has been shut down
INFO 2017-05-19 12:49:43,631 - Started thread pool with 3 core threads and 20 maximum threads
INFO 2017-05-19 12:49:43,631 - [AlertScheduler] Stopped the alert scheduler.
INFO 2017-05-19 12:49:43,632 - Shutting down thread pool
INFO 2017-05-19 12:49:43,639 - Server connection disconnected.

On the ambari-server side there is only such log at the same time:

19 maj 2017 12:49:41,621  WARN [qtp-ambari-agent-43] SecurityFilter:103 - Request https://hadoop-metrics.pip:8440/ca doesn't match any pattern.
19 maj 2017 12:49:41,621  WARN [qtp-ambari-agent-43] SecurityFilter:62 - This request is not allowed on this port: https://hadoop-metrics.pip:8440/ca

It can't be networking problem - it's on the same node. Ambari-agents on other nodes behave the same way.

I also gave a try of installing ambari-server on a completely fresh server with Ubuntu16 following instructions from

It gave me the same kind of errors. Maybe this can help in finding the cause of this problem.


@Darek Rutkowski

Okay now to validate the process,do the following


  1. Install the Ambari Agent on every host in your cluster.

    apt-get install ambari-agent

  2. Configure the Ambari Agent by editing the ambari-agent.ini file as shown in the following example:

    vi /etc/ambari-agent/conf/ambari-agent.ini





  3. Start the agents on every host in your cluster.

    ambari-agent start

    The agents should now register with the Ambari server <your.ambari.server.hostname> and this should be the FQDN of the Ambari server


Sorry for delay, I had a week off...

Thanks for your hints and engagement.

I decided to roll back to Ambari version 2.2.2 to take back control over systems. In a few weeks I will give it another try.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.