Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ambari-server start error

ambari-server start error

New Contributor

I am trying to setup a cluster with HDP 3.1. I have a server with clean CentOS 7.4. I installed ambari 2.7.3 using this document. https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/set_up_the_a... 

After completed all steps, I got error when try to start ambari server. 

 

[root@msl-dpe-perf80-100g resources]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? 
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...

Do you want to change Oracle JDK [y/n] (n)? Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (localhost): msl-dpe-perf80-100g.msl.lab
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (bigdata): 
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? 
Extracting system views...
.....
Ambari repo file contains latest json url <a href="http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json" target="_blank">http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json</a>, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
[root@msl-dpe-perf80-100g resources]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start........................................ERROR: Exiting with exit code -1. 
REASON: Ambari Server java process has stopped. Please check the logs for more information.
[root@msl-dpe-perf80-100g resources]# 

 

The error message reported in ambari-server.log is:

 

019-08-21 15:30:03,832  INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.ViewInstanceEventCreator
2019-08-21 15:30:03,832  INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.ViewPrivilegeEventCreator
2019-08-21 15:30:04,856  INFO [main] HostRoleCommandDAO:277 - Host role command status summary cache enabled !
2019-08-21 15:30:04,858  INFO [main] TransactionalLock$LockArea:121 - LockArea HRC_STATUS_CACHE is enabled
2019-08-21 15:30:05,081  INFO [main] AmbariServerConfigurationProvider:68 - Registered org.apache.ambari.server.ldap.service.AmbariLdapConfigurationProvider in event publisher
2019-08-21 15:30:05,584  INFO [main] CredentialStoreServiceImpl:60 - Initialized the temporary credential store. KeyStore entries will be retained for 90 minutes and will be actively purged
2019-08-21 15:30:05,593  INFO [main] LockFactory:54 - Lock profiling is disabled
2019-08-21 15:30:05,594  INFO [main] AmbariServer:1083 - Getting the controller
2019-08-21 15:30:06,282  INFO [pool-5-thread-1] AmbariServerConfigurationProvider:98 - JPA initialized event received: JpaInitializedEvent{eventType=JPA_INITIALIZED}
2019-08-21 15:30:06,282  INFO [pool-5-thread-1] AmbariServerConfigurationProvider:122 - Loading ldap-configuration configuration data
2019-08-21 15:30:06,377  INFO [main] AbstractPoolBackedDataSource:212 - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 5, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 2vsgnka41ekyetann3ejj|57bc27f5, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> com.mysql.jdbc.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 2vsgnka41ekyetann3ejj|57bc27f5, idleConnectionTestPeriod -> 7200, initialPoolSize -> 5, jdbcUrl -> jdbc:mysql://msl-dpe-perf80-100g.msl.lab:3306/ambari, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 14400, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 32, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> SELECT 1, privilegeSpawnedThreads -> false, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
2019-08-21 15:30:36,719  WARN [C3P0PooledConnectionPoolManager[identityToken->2vsgnka41ekyetann3ejj|57bc27f5]-HelperThread-#1] BasicResourcePool:223 - com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@30120b4 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
        at sun.reflect.GeneratedConstructorAccessor48.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
        at com.mysql.jdbc.Util.getInstance(Util.java:386)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1015)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2570)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2306)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:839)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
        at sun.reflect.GeneratedConstructorAccessor42.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:421)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:350)
        at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
        at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
        at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
        at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)

 

mysql connector was installed as:

 

harry.li@msl-dpe-perf80:~$  sudo yum install mysql-connector-java*
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * epel: d2lzkl7pfhq30w.cloudfront.net
Resolving Dependencies
--> Running transaction check
---> Package mysql-connector-java.noarch 1:5.1.25-3.el7 will be installed
--> Processing Dependency: jta >= 1.0 for package: 1:mysql-connector-java-5.1.25-3.el7.noarch
--> Processing Dependency: java >= 1:1.6.0 for package: 1:mysql-connector-java-5.1.25-3.el7.noarch
--> Processing Dependency: slf4j for package: 1:mysql-connector-java-5.1.25-3.el7.noarch
--> Processing Dependency: jpackage-utils for package: 1:mysql-connector-java-5.1.25-3.el7.noarch
--> Running transaction check
---> Package geronimo-jta.noarch 0:1.1.1-17.el7 will be installed
---> Package java-1.8.0-openjdk.x86_64 1:1.8.0.161-0.b14.el7_4 will be installed
--> Processing Dependency: java-1.8.0-openjdk-headless(x86-64) = 1:1.8.0.161-0.b14.el7_4 for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: xorg-x11-fonts-Type1 for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libjvm.so(SUNWprivate_1.1)(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libjli.so(SUNWprivate_1.1)(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libjava.so(SUNWprivate_1.1)(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libjvm.so()(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libjli.so()(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libjava.so()(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libgif.so.4()(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
--> Processing Dependency: libawt.so()(64bit) for package: 1:java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
---> Package javapackages-tools.noarch 0:3.4.1-11.el7 will be installed
--> Processing Dependency: python-javapackages = 3.4.1-11.el7 for package: javapackages-tools-3.4.1-11.el7.noarch
--> Processing Dependency: libxslt for package: javapackages-tools-3.4.1-11.el7.noarch
---> Package slf4j.noarch 0:1.7.4-4.el7_4 will be installed
--> Processing Dependency: mvn(log4j:log4j) for package: slf4j-1.7.4-4.el7_4.noarch
--> Processing Dependency: mvn(javassist:javassist) for package: slf4j-1.7.4-4.el7_4.noarch
--> Processing Dependency: mvn(commons-logging:commons-logging) for package: slf4j-1.7.4-4.el7_4.noarch
--> Processing Dependency: mvn(commons-lang:commons-lang) for package: slf4j-1.7.4-4.el7_4.noarch
--> Processing Dependency: mvn(ch.qos.cal10n:cal10n-api) for package: slf4j-1.7.4-4.el7_4.noarch
--> Running transaction check
---> Package apache-commons-lang.noarch 0:2.6-15.el7 will be installed

 

 

3 REPLIES 3
Highlighted

Re: ambari-server start error

Explorer

So the exception you have listed indicates that the JDBC driver was installed properly, since the callstack includes code in the MySQL driver. I would check that the hostname and port you supplied are correct, and that you can access that node from the node you are working on. The commands below would be a good starting point.

ping msl-dpe-perf80-100g.msl.lab 
telnet msl-dpe-perf80-100g.msl.lab 3306

Re: ambari-server start error

Cloudera Employee

@harry_li - Can you check and confirm, if your MYSQL is up and running? Also refer below article which talks about the error message which you are getting:

 

https://community.cloudera.com/t5/Support-Questions/ambari-2-7-0-not-start/m-p/198737

Re: ambari-server start error

Explorer

First check if the database has been started.

# check mysql status
systemctl status

# start mysql if it didn't started 
systemctl start mysqld

Then set up remote access for the mysql user.

#login 
mysql -uroot -p

# set up remote access 
grant all on *.* to 'root'@'localhost' identified by '<your_pwd>'

Restart ambari again.