Created 01-25-2017 12:49 AM
I have hit the following error while upgrading the ambari and HDP stack
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff} span.s1 {font-variant-ligatures: no-common-ligatures}
2017-01-24 15:05:48,420 ERROR - Unexpected error, database check failed
java.lang.NullPointerException
at org.apache.ambari.server.checks.DatabaseConsistencyCheckHelper.checkServiceConfigs(DatabaseConsistencyCheckHelper.java:543)
at org.apache.ambari.server.checks.DatabaseConsistencyChecker.main(DatabaseConsistencyChecker.java:115)
I have started the ambari-server by -skip-database-check.
But, is it good to proceed with the upgrade? What are the steps that I need to follow to continue with the upgrade?
Created 01-25-2017 01:03 AM
It is advised to correct Database consistency issues before HDP upgrade. looks like some of the services are having inconsistency. what is the exact Ambari version are you using?
Created 01-25-2017 01:12 AM
@Bipinkumar Algubelli I'd suggest fixing the ambari database inconsistency first prior to upgrade else it may cause issues during upgrade. You can find leads about inconsistency at /var/log/ambari-server/ambari-server-check-database.log from Ambari server.
Created 01-25-2017 01:37 AM
@apappu My previous ambari version was 2.2. I have just upgraded it to 2.4
Created 01-25-2017 01:43 AM
@apatel I am unable to figure out the issue from /var/log/ambari-server/ambari-server-check-database.log
2017-01-24 17:25:35,866 INFO - ******************************* Check database started ******************************* 2017-01-24 17:25:40,576 INFO - Checking for configs not mapped to any cluster 2017-01-24 17:25:40,600 INFO - Checking for configs selected more than once 2017-01-24 17:25:40,603 INFO - Checking for hosts without state 2017-01-24 17:25:40,604 INFO - Checking host component states count equals host component desired states count 2017-01-24 17:25:40,606 INFO - Checking services and their configs 2017-01-24 17:25:42,908 ERROR - Unexpected error, database check failed java.lang.NullPointerException at org.apache.ambari.server.checks.DatabaseConsistencyCheckHelper.checkServiceConfigs(DatabaseConsistencyCheckHelper.java:543) at org.apache.ambari.server.checks.DatabaseConsistencyChecker.main(DatabaseConsistencyChecker.java:115)
Created 01-25-2017 01:49 AM
@apappu @apatel I searched and found that this is a know issue https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.0.1/bk_ambari-release-notes/content/ambari_reln.... I skipped the database check. But I am worried about the upgrade
Created 01-25-2017 02:54 AM
throwing NullPointerexception is known issue - but you will have to find out why it is throwing that exception and have to fix that first.
you can run below query and find out which service it is.
SELECT c.cluster_name, cs.service_name, cc.type_name, sc.version FROM clusterservices cs JOIN serviceconfig sc ON cs.service_name = sc.service_name AND cs.cluster_id = sc.cluster_id JOIN serviceconfigmapping scm ON sc.service_config_id = scm.service_config_id JOIN clusterconfig cc ON scm.config_id = cc.config_id AND sc.cluster_id = cc.cluster_id JOIN clusters c ON cc.cluster_id = c.cluster_id AND sc.stack_id = c.desired_stack_id WHERE sc.group_id IS NULL AND sc.service_config_id = (SELECT MAX(service_config_id) FROM serviceconfig sc2 WHERE sc2.service_name = sc.service_name AND sc2.cluster_id = sc.cluster_id) GROUP BY c.cluster_name, cs.service_name, cc.type_name, sc.version
Created 01-25-2017 06:32 PM
@apappu I tried the query and it gave me 103 rows of result. Should I remove all those configurations?
Created 01-25-2017 06:53 PM
I am sorry - please ignore my previous query. can you please upgrade your Ambari to 2.4.2 - then try to start the Ambari server and check /var/log/ambari-server/ambari-server-check-database.log log for error messages. based on the error message you can check hostcomponfrom that you will have to check some of the services in hostcomponentstate table and fix the issues.
Created 02-17-2017 07:55 AM
Bipinkumar Algubelli first try to upgrade ambari version to 2.4.1.0 & then try to upgrade HDP to 2.4. There are manual steps which you can follow to upgrade to 2.4 as well as the docs provided at