Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Install and configure new Ambari server for existing HDF cluster

avatar
Rising Star

Hi, all!

Can anyone give any suggestion in configuring new Ambari server with an existing HDF cluster

Previos version of ambari fails in ambari-server start after downgrade

[root@bi-nifi1 dmp]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.............ERROR: Exiting with exit code -1. 
REASON: Ambari Server java process has stopped. Please check the logs for more information.
[root@bi-nifi1 dmp]# cat /var/log/ambari-server/ambari-server.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
An unexpected error occured during starting Ambari Server.
org.apache.ambari.server.AmbariException: An error occured during updating current repository versions with stack repositories.
        at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:99)
        at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
        at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1061)
Caused by: java.lang.NullPointerException
        at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.updateRepoVersion(UpdateActiveRepoVersionOnStartup.java:106)
        at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:92)
        ... 2 more
Exception in thread "main" org.apache.ambari.server.AmbariException: Error stopping the server
        at org.apache.ambari.server.controller.AmbariServer.stop(AmbariServer.java:880)
        at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1075)
[root@bi-nifi1 dmp]# 
1 ACCEPTED SOLUTION

avatar
Master Collaborator

@Gulshan Agivetova

From the ambari-server log, I can see Ambari could not load version definition for HDP-2.6. I would recommended you to have clean all python libraries, yum cleanup and freshly reinstall. Could you please try below steps. appropriate to your host machine.

  • Clean Ambari old installation which include cleaning up the old Python libraries installation
yum remove ambari-server ambari-agent -y
rm -f /usr/sbin/ambari*
rm -f /usr/lib/python2.6/site-packages/ambari_commons
rm -rf /usr/lib/python2.6/site-packages/resource_management
rm -rf /usr/lib/python2.6/site-packages/ambari_jinja2
rm -rf /usr/lib/ambari-server
rm -rf /usr/lib/ambari-agent
  • Get the new repo according to your host machine
wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.5.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
  • Perform a yum cleanup
yum clean all
  • Now do a fresh install of ambari binaries.
yum install ambari-server -y
yum install ambari-agent -y
  • Now perform the ambari-server setup again.
/usr/sbin/ambari-server.py setup --databasehost=localhost --databasename=ambari --databaseusername=ambari --postgresschema=ambari --databasepassword=ambari --databaseport=5432 --database=postgres -s

I guess these above steps may resolve your issue.

View solution in original post

7 REPLIES 7

avatar
Master Collaborator

Could you please attach ambari-server.log for more detail about error ?

avatar
Rising Star

avatar
Master Collaborator

@Gulshan Agivetova

This seems to be related to some uncleaned installation. Mostly the "/usr/lib/ambari-server/lib/ambari_commons" and "/usr/lib/python2.6/site-packages/ambari_commons" might have some older version of libraries installation. So please clean up those old python libraries. Added to that setup again ambari-server using ambari-server.py script appropriate to your cluster configuration.

/usr/sbin/ambari-server.py setup --databasehost=localhost --databasename=ambari --databaseusername=ambari --postgresschema=ambari --databasepassword=ambari --databaseport=5432 --database=postgres -s

avatar
Rising Star

Hi @Jagadeesan A S

Thank you for your concern!

How can I identify which libraries are from older versions, there is no version on the file name?

[dmp@bi-nifi1 ~]$ cd /usr/lib/ambari-server/lib/ambari_commons
[dmp@bi-nifi1 ambari_commons]$ ls
aggregate_functions.py     ambari_service.pyc           credential_store_helper.pyo  inet_utils.py     logging_utils.pyc  os_check.pyo        os_utils.py             parallel_processing.pyc  str_utils.pyc        xml_utils.pyo
aggregate_functions.pyc    ambari_service.pyo           exceptions.py                inet_utils.pyc    logging_utils.pyo  os_family_impl.py   os_utils.pyc            parallel_processing.pyo  str_utils.pyo        yaml_utils.py
aggregate_functions.pyo    constants.py                 exceptions.pyc               inet_utils.pyo    network.py         os_family_impl.pyc  os_utils.pyo            resources                urllib_handlers.py   yaml_utils.pyc
ambari_metrics_helper.py   constants.pyc                exceptions.pyo               __init__.py       network.pyc        os_family_impl.pyo  os_windows.py           shell.py                 urllib_handlers.pyc  yaml_utils.pyo
ambari_metrics_helper.pyc  constants.pyo                firewall.py                  __init__.pyc      network.pyo        os_linux.py         os_windows.pyc          shell.pyc                urllib_handlers.pyo
ambari_metrics_helper.pyo  credential_store_helper.py   firewall.pyc                 __init__.pyo      os_check.py        os_linux.pyc        os_windows.pyo          shell.pyo                xml_utils.py
ambari_service.py          credential_store_helper.pyc  firewall.pyo                 logging_utils.py  os_check.pyc       os_linux.pyo        parallel_processing.py  str_utils.py             xml_utils.pyc










avatar
Master Collaborator

@Gulshan Agivetova

From the ambari-server log, I can see Ambari could not load version definition for HDP-2.6. I would recommended you to have clean all python libraries, yum cleanup and freshly reinstall. Could you please try below steps. appropriate to your host machine.

  • Clean Ambari old installation which include cleaning up the old Python libraries installation
yum remove ambari-server ambari-agent -y
rm -f /usr/sbin/ambari*
rm -f /usr/lib/python2.6/site-packages/ambari_commons
rm -rf /usr/lib/python2.6/site-packages/resource_management
rm -rf /usr/lib/python2.6/site-packages/ambari_jinja2
rm -rf /usr/lib/ambari-server
rm -rf /usr/lib/ambari-agent
  • Get the new repo according to your host machine
wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.5.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
  • Perform a yum cleanup
yum clean all
  • Now do a fresh install of ambari binaries.
yum install ambari-server -y
yum install ambari-agent -y
  • Now perform the ambari-server setup again.
/usr/sbin/ambari-server.py setup --databasehost=localhost --databasename=ambari --databaseusername=ambari --postgresschema=ambari --databasepassword=ambari --databaseport=5432 --database=postgres -s

I guess these above steps may resolve your issue.

avatar
Rising Star

Hi @Jagadeesan A S

Thank you for your response!

I have followed all the steps you have stated. After ambari-server setup I dropped the new ambari database, and recovered the database from backup.

And when trying to start ambari-server it gives following error:

ServiceNotFoundException: Service not found, clusterName=hdf_kg, serviceName=AMBARI_INFRA
# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start................
DB configs consistency check found warnings. See /var/log/ambari-server/ambari-server-check-database.log for more details.
ERROR: Exiting with exit code -1. 
REASON: Ambari Server java process has stopped. Please check the logs for more information.
[root@bi-nifi1 backup]# cat /var/log/ambari-server/ambari-server.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
Database consistency check result: DB_CHECK_WARNING
An unexpected error occured during starting Ambari Server.
org.apache.ambari.server.ServiceNotFoundException: Service not found, clusterName=hdf_kg, serviceName=AMBARI_INFRA
        at org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:864)
        at org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1240)
        at org.apache.ambari.server.controller.AmbariServer.run(AmbariServer.java:532)
        at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1067)
[root@bi-nifi1 backup]# 

Is it possible to manually start AMBARI_INFRA service?


Thanks in advance!

avatar
Master Collaborator

@Gulshan Agivetova

You can force Ambari Server to start by skipping this check with the following option:

ambari-server start --skip-database-check