how to resolve the DB configs consistency check failed

[#1]

hi all

we have a problem to start the ambari server

we try to start the ambari server after removing some services from ambari

for now we get the error as

DB configs consistency check failed. Run "ambari-server start --skip-database-check" to skip. You may try --auto-fix-database flag to attempt to fix issues automatically. If you use this "--skip-database-check" option, do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See /var/log/ambari-server/ambari-server-check-database.log for more details on the consistency issues.
ERROR: Exiting with exit code -1.
REASON: Ambari Server java process has stopped. Please check the logs for more information.
You have mail in 

we try also the flag --auto-fix-database ( but not help )

[root@master ~]# ambari-server restart
Using python  /usr/bin/python
Restarting ambari-server
Waiting for server stop...
Ambari Server stopped
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start............
DB configs consistency check failed. Run "ambari-server start --skip-database-check" to skip. You may try --auto-fix-database flag to attempt to fix issues automatically. If you use this "--skip-database-check" option, do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See /var/log/ambari-server/ambari-server-check-database.log for more details on the consistency issues.
ERROR: Exiting with exit code -1.
REASON: Ambari Server java process has stopped. Please check the logs for more information.
You have mail in /var/spool/mail/root




[root@master ~]# ambari-server restart --auto-fix-database
Using python  /usr/bin/python
Restarting ambari-server
Ambari Server is not running
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start............
DB configs consistency check failed. Run "ambari-server start --skip-database-check" to skip. You may try --auto-fix-database flag to attempt to fix issues automatically. If you use this "--skip-database-check" option, do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See /var/log/ambari-server/ambari-server-check-database.log for more details on the consistency issues.
ERROR: Exiting with exit code -1.
REASON: Ambari Server java process has stopped. Please check the logs for more information.




[root@master ~]# ambari-server start --skip-database-check
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari Server is starting with the database consistency check skipped. Do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See "/var/log/ambari-server/ambari-server-check-database.log" for more details on the consistency issues.
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.........................
Server started listening on 8080
Ambari Server 'start' completed successfully.

and finally we not have choice and we start the ambari with skip --database check

ambari-server start --skip-database-check

so what the steps that neded in order to fix this isshue ?

[#2]

Hi @Michael Bronson,

Can you please in /var/log/ambari-server/ambari-server-check-database.log and see what exception you are hitting.

You can refer to following article if you are having same warning : https://community.hortonworks.com/articles/227486/how-to-fix-database-consistency-warnings-in-ambari...

If this is not the warning, please give the server log snippet in this thread and tag me too 🙂

If this comment helps you please accept my answer

[#3]

regarding the article - what is the login/password before running SELECT commands and how to get the SHELL ?

[#4]

in case we have the following warning , then how should be the commands of SELECT ?

<br>2018-12-20 20:32:50,045  WARN - You have config(s): [LOT_SER, GOP_SER] that is(are) not mapped (in clusterconfigmapping table) to any cluster!
2018-12-20 20:32:50,061  WARN - Service BON_SER is not available for stack HDP-2.6 in cluster HDP

[#5]

Hi @Michael Bronson,

You need to login to ambari database to perform the steps mentioned in the article.

If your database is postgresql you need to login like this :

[root@alatestambari1 tmp]# psql -U ambari ambari
Password for user ambari: <default password is bigdata>

and execute the commands as mentioned there.

If you are using non-default databases it depends on your database :

for ex: mysql

[root@alatestambari1 tmp]#mysql -u ambari ambari
password: <password of user ambari>
ambari=> use ambari;

Please see if this is helpful.

[#6]

about the warning as :

what should be the releb]vant SQL command?

LOT_SER, GOP_SER , BON_SER <-- they are services

<br>2018-12-20 20:32:50,045  WARN - You have config(s): [LOT_SER, GOP_SER] that is(are) not mapped (in clusterconfigmapping table) to any cluster!
2018-12-20 20:32:50,061  WARN - Service BON_SER is not available for stack HDP-2.6 in cluster HDP

[#7]

I have also this warning - maybe this is more relevant

WARN - You have config(s): MOON_SERVICE-version1530186236790,MOON_SERVICE-version1530714719817475,MOON_SERVICE-version1530714831573,MOON_SERVICE-version1530174193259482,JWE_SER-version1545298738838,GFT_SER-version1529923094645387,MOON_SERVICE-version1529922706468805,MOON_SERVICE-version1530108940699945,MOON_SERVICE-version1531392078026,GFT_SER-version1545298738838 that is(are) not mapped (in serviceconfigmapping table) to any service!

while the article talk about this warning

WARN - You have config(s): webhcat-site-version1540591929056,webhcat-log4j-version1540591928580,hive-exec-log4j-version1540591929168,webhcat-env-version1540591929289,hive-log4j-version1540591929465,hcat-env-version1540591928892 that is(are) not mapped (in serviceconfigmapping table) to any service!

[#8]

Hi @Michael Bronson,,

Basically this error happens due to some orphan configs in your database that is not linked to any service.

you can ignore those warnings if required or fix it as mentioned in article abouve.

you can take a backup of your database if you are worried to do it.

and yes these warnings are related .

Please accept answer if it helped.

[#9]

meanwhile I cant ignore because the ambari server not start , so I need to fix it

since article not slimier to my warning , I just want to know how to do as explained in the article but change the syntax according to the warning as get

[#10 · Accepted solution]

Hi @Michael Bronson ,

refer to this article : https://community.hortonworks.com/articles/227486/how-to-fix-database-consistency-warnings-in-ambari...

The solution given here is irrespective of any config, it will work 100% for your DB warning too.

its taking backup of your orphaned configs and making the value mapped=1.

Please let me know if it doesnt work.

You can take backup if you are afraid to do so.