Looks like one of your host is causing the problem.
Please check the following:
1). Check the "host_version" table and findout which host is not pointing to "22.214.171.124-37"
SELECT * FROM host_version;
2). Login to the problematic host using SSH and then run the following command to find out if it really has the 126.96.36.199-37 packages installed or not?
3). If you see that the hdp-select command output shows that all the packages installed to that host is having the correct version then following the next step.
4). If you find the problematic host then find it's host_id from the "hosts" table and then check if it is pointing to OLD repo_version_id , if yes then update it to point to the current version.
UPDATE host_version SET state='CURRENT' WHERE host_id='xx';
Also you can update the "repo_version_id" for this host to point to the latest version of repo (if it is not set already)
Please make sure that after this change the host is pointing to the repo_version_id mentioned in the "cluster_version" table which has state as "CURRENT". .
5). After the DB changes (commits) ambari-server need to be restarted.
# ambari-server restart
Thank you Jay.
1. I checked all host hdp-select, all versions are the same.
2. even if I find one out of sync, I dont know how to run this SELECT * FROM host_version; I guess I install oracle database for metadata, not mysql, please let me know if I am wrong.
I just installed one more zookeeper, it allowed me to add new service. I know it dont make sense, but that is only thing I did.
I think you provide all right answers. just let me know how to this query SELECT * FROM host_version;
thank you again.
You can check the "/etc/ambari-server/conf/ambari.properties" file to know exactly which DataBase your ambari is using and then accordingly you can login to the DB host and run/check the requested commands or the DB admin can assist with the queries.
Example: (In my Case it is Postgres)
# grep 'jdbc' /etc/ambari-server/conf/ambari.properties custom.postgres.jdbc.name=postgresql-9.3-1101-jdbc4.jar previous.custom.postgres.jdbc.name=postgresql-9.3-1101-jdbc4.jar server.jdbc.connection-pool=internal server.jdbc.database=postgres server.jdbc.database_name=ambari server.jdbc.postgres.schema=ambari server.jdbc.user.name=ambari server.jdbc.user.passwd=/etc/ambari-server/conf/password.dat