Community Articles
Find and share helpful community-sourced technical articles
Labels (1)
Super Guru

After completing Ambari upgrade from Ambari 1.7 to Ambari 2.0.2, Ambari failed to start because of below error(s):

[root@xyz ~]# service ambari-server start 
Using python /usr/bin/python2.6 
Starting ambari-server 
Ambari Server running with administrator privileges. 
Organizing resource files at /var/lib/ambari-server/resources... 
Server PID at: /var/run/ambari-server/ambari-server.pid 
Server out at: /var/log/ambari-server/ambari-server.out 
Server log at: /var/log/ambari-server/ambari-server.log 
Waiting for server start.................... 
ERROR: Exiting with exit code -1. 
REASON: Ambari Server java process died with exitcode 255. Check /var/log/ambari-server/ambari-server.out for more information. 

.

If you check /var/log/ambari-server/ambari-server.out logs, you get below error(s):

[root@xyz ~]# cat /var/log/ambari-server/ambari-server.out 
[EL Warning]: metadata: 2016-12-01 10:01:23.436--ServerSession(306824177)--The reference column name [resource_type_id] mapped on the element [field permissions] does not correspond to a valid id or basic field/column on the mapping reference. Will use referenced column name as provided. 
[EL Info]: 2016-12-01 10:01:24.412--ServerSession(306824177)--EclipseLink, version: Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd 
[EL Info]: connection: 2016-12-01 10:01:24.603--ServerSession(306824177)--file:/usr/lib/ambari-server/ambari-server-2.0.2.25.jar_ambari-server_url=jdbc:postgresql://localhost/ambari_user=ambari login successful

.

Note - Above error is complex and confusing. You should always check /var/log/ambari-server/ambari-server.log.

In my case, I was getting below error in ambari-server.log

org.apache.ambari.server.StackAccessException: Stack data, stackName=HDP, stackVersion=2.2, serviceName=SMARTSENSE
        at org.apache.ambari.server.api.services.AmbariMetaInfo.getService(AmbariMetaInfo.java:497)
        at org.apache.ambari.server.api.services.AmbariMetaInfo.getComponent(AmbariMetaInfo.java:265)
        at org.apache.ambari.server.controller.utilities.DatabaseChecker.checkDBConsistency(DatabaseChecker.java:96)
        at org.apache.ambari.server.controller.AmbariServer.run(AmbariServer.java:217)
        at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:665)

.

Why this has happened?

This is one of the Known issue - Sometimes stack component for SmartSense gets removed from Ambari DB after an upgrade.

.

How to resolve this?

1. Check what version of SmartSense you have installed on your cluster.

2. Download RPM from Customer Support portal

3. Run below command to re-add the SmartSense view and Stack to the Ambari DB.

[root@xyz crazyadmins.com]# hst add-to-ambari
Enter SmartSense distributable path: /home/kuldeepk/smartsense-hst-1.2.1.0-161.x86_64.rpm
Added SmartSense service definition to Ambari

4. Restart 'ambari-server'.

[root@xyz crazyadmins.com]# ambari-server restart
Using python  /usr/bin/python
Restarting ambari-server
Using python  /usr/bin/python
Stopping ambari-server
Ambari Server is not running
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Ambari Server 'start' completed successfully.
[root@prodnode1 ~]#

.

Please comment if you have any feedback/questions/suggestions. Happy Hadooping!! :)

464 Views