PROBLEM:
Ambari Server won't be able to start because of DB inconsistencies.
Sample Error:
2017-02-06 05:08:43,975 ERROR - You have non selected configs: zeppelin-ambari-config for service ZEPPELIN from cluster XXXX! 2017-02-06 05:08:43,976 INFO - ******************************* Check database completed ******************************* 2017-02-06 05:10:12,834 INFO - Checking DB store version 2017-02-06 05:10:14,094 INFO - DB store version is compatible 2017-02-07 13:50:31,769 INFO - ******************************* Check database started ******************************* 2017-02-07 13:50:41,247 INFO - Checking for configs not mapped to any cluster 2017-02-07 13:50:41,322 INFO - Checking for configs selected more than once 2017-02-07 13:50:41,326 INFO - Checking for hosts without state 2017-02-07 13:50:41,330 INFO - Checking host component states count equals host component desired states count 2017-02-07 13:50:41,334 INFO - Checking services and their configs 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / SQOOP 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / HDFS 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / MAPREDUCE2 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / TEZ 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / SPARK 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / HBASE 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / ZOOKEEPER 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / YARN 2017-02-07 13:50:45,793 INFO - Processing HDP-2.5 / KNOX 2017-02-07 13:50:45,794 INFO - Processing HDP-2.5 / PIG 2017-02-07 13:50:45,794 INFO - Processing HDP-2.5 / RANGER 2017-02-07 13:50:45,794 INFO - Processing HDP-2.5 / HIVE 2017-02-07 13:50:45,794 INFO - Processing HDP-2.5 / SLIDER 2017-02-07 13:50:45,794 INFO - Processing HDP-2.5 / AMBARI_INFRA 2017-02-07 13:50:45,794 INFO - Processing HDP-2.5 / KAFKA 2017-02-07 13:50:45,794 INFO - Processing HDP-2.5 / SMARTSENSE 2017-02-07 13:50:45,809 ERROR - You have non selected configs: zeppelin-ambari-config for service ZEPPELIN from cluster XXXXX! 2017-02-07 13:50:45,810 INFO - ******************************* Check database completed *******************************
.
BUSINESS IMPACT:
It's not recommended to make any changes in service configurations because backend Database is not consistent.
.
WORKAROUND:
ambari-server start --skip-database-check
Note - This is not recommended for production clusters, if you do this, please do not make any modifications in service configurations till you resolves the conflicts.
.
RESOLUTION:
1. Stop Ambari server
ambari-server stop
.
2. Take backup of Ambari Database
For postgres - Use pg_dump command.
For MySql - Use mysqldump command.
.
3. Run below queries to resolve conflicts
delete from hostcomponentstate where service_name = 'ZEPPELIN'; delete from hostcomponentdesiredstate where service_name = 'ZEPPELIN'; delete from servicecomponentdesiredstate where service_name = 'ZEPPELIN'; delete from servicedesiredstate where service_name = 'ZEPPELIN'; delete from serviceconfighosts where service_config_id in (select service_config_id from serviceconfig where service_name = 'ZEPPELIN'); delete from serviceconfigmapping where service_config_id in (select service_config_id from serviceconfig where service_name = 'ZEPPELIN'); delete from serviceconfig where service_name = 'ZEPPELIN'; delete from requestresourcefilter where service_name = 'ZEPPELIN'; delete from requestoperationlevel where service_name = 'ZEPPELIN'; delete from clusterservices where service_name ='ZEPPELIN'; delete from clusterconfig where type_name like 'zeppelin%'; delete from clusterconfigmapping where type_name like 'zeppelin%';
.
4. Start Ambari Server and it should come up without any inconsistencies.
.
