Support Questions
Find answers, ask questions, and share your expertise

postgresql + ambari server failed to open port 8080

Solved Go to solution

postgresql + ambari server failed to open port 8080

We have ambari cluster

we have issue with ambari server

example

ambari-server start

Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.Organizing resource files at /var/lib/ambari-server/resources...Ambari database consistency check started...Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start............................................................ERROR: Exiting with exit code 1.REASON: Server not yet listening on http port 8080 after 50 seconds. Exiting.

ambari server create the PID and service is up but server not listening on port 8080

we check the firewall and iptables but this services are down

we check the PostgreSQL service ( is running well )

systemctl status postgresql.service postgresql.service - PostgreSQL database server
   Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: disabled)   Active: active (running) since Thu 2020-01-30 12:49:26 UTC; 20min ago

we also checked the following

cat /etc/ambari-server/conf/password.dat
bigdata

grep 'ambari' /var/lib/pgsql/data/pg_hba.conf
local  all  ambari,mapred md5
host  all   ambari,mapred 0.0.0.0/0  md5
host  all   ambari,mapred ::/0 md5



grep 'jdbc'  /etc/ambari-server/conf/ambari.properties
custom.postgres.jdbc.name=postgresql-jdbc.jar
previous.custom.postgres.jdbc.name=postgresql-jdbc.jar
server.jdbc.connection-pool=internal
server.jdbc.database=postgres
server.jdbc.database_name=ambari
server.jdbc.postgres.schema=ambari
server.jdbc.user.name=ambari

but seems they are ok

so what could be the reason that the default port 8080 of ambari server isn’t opened

30 Jan 2020 12:51:50,152 ERROR [main] DBAccessorImpl:119 - Error while creating database accessor
org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:207)        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)        at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:138)        at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)        at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)        at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31)        at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)        at org.postgresql.Driver.makeConnection(Driver.java:410)        at org.postgresql.Driver.connect(Driver.java:280)        at java.sql.DriverManager.getConnection(DriverManager.java:664)        at java.sql.DriverManager.getConnection(DriverManager.java:247)        at org.apache.ambari.server.orm.DBAccessorImpl.<init>(DBAccessorImpl.java:93)        at org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance(<generated>)        at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)        at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)        at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)        at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)        at com.google.inject.Scopes$1$1.get(Scopes.java:65)        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)        at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)        at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)        at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)        at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)        at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)        at com.google.inject.Scopes$1$1.get(Scopes.java:65)        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)        at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)        at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)        at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)        at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)        at com.google.inject.Scopes$1$1.get(Scopes.java:65)        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)        at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)        at com.google.inject.internal.InjectionRequestProcessor$StaticInjection$1.call(InjectionRequestProcessor.java:116)        at com.google.inject.internal.InjectionRequestProcessor$StaticInjection$1.call(InjectionRequestProcessor.java:110)        at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)        at com.google.inject.internal.InjectionRequestProcessor$StaticInjection.injectMembers(InjectionRequestProcessor.java:110)        at com.google.inject.internal.InjectionRequestProcessor.injectMembers(InjectionRequestProcessor.java:78)        at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:170)        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)        at com.google.inject.Guice.createInjector(Guice.java:95)        at com.google.inject.Guice.createInjector(Guice.java:72)        at com.google.inject.Guice.createInjector(Guice.java:62)        at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1045)Caused by: java.net.ConnectException: Connection timed out (Connection timed out)        at java.net.PlainSocketImpl.socketConnect(Native Method)        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)        at java.net.Socket.connect(Socket.java:589)        at java.net.Socket.connect(Socket.java:538)        at org.postgresql.core.PGStream.<init>(PGStream.java:60)        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:101)
        ... 52 more

 

Michael-Bronson
1 ACCEPTED SOLUTION

Accepted Solutions

Re: postgresql + ambari server failed to open port 8080

Dear Jay

 

finally we found the issue

 

it was about the mistake in /etc/hosts file

 

instead of 127.0.0.1 , it was the ip address - 27.0.0.1

 

so we fix it

 

and restart the postgresql and ambari

 

now all are fine 

 

 

Michael-Bronson

View solution in original post

32 REPLIES 32

Re: postgresql + ambari server failed to open port 8080

Super Mentor

@mike_bronson7 

Can you please try the following command to verify if Postgres is listening to * address and port is opened?

# ps -ef | grep pgsql | grep ^postgres
# netstat -tnlpa | grep 5432
# netstat -tnlpa | grep $POSTGRES_PID

If above does not show the port is listening properly then we might need to check the postgres logs.

Next can you please check if the Following Test Postgres DB connection command is running without any issue on Ambari Server host?

# /usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-42.2.2.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://localhost:5432/ambari ambari bigdata org.postgresql.Driver



The please try to run the same command using HOSTNAME of PostgresDB (instead of "localhost")

# /usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-42.2.2.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://localhost:5432/ambari ambari bigdata org.postgresql.Driver


Please NOTE that the "/usr/lib/ambari-server/postgresql-42.2.2.jar" PATH and the JDK installation path might be different in your case so please update the command accordingly.

 

.

.
Also as we see the message :

REASON: Server not yet listening on http port 8080 after 50 seconds. Exiting.


So it will be also good to try setting the following and then try to restart ambari again properto to a little high value like 120 or 150 seconds as sometimes ambari takes little longer time to start. Default value is 50 seconds.

server.startup.web.timeout=150

 

.

Re: postgresql + ambari server failed to open port 8080

Dear Jay

 

here the first test

 

/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-42.2.2.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://localhost:5432/ambari ambari bigdata org.postgresql.Driver
ERROR: Unable to connect to the DB. Please check DB connection properties.
java.lang.ClassNotFoundException: org.postgresql.Driver

Michael-Bronson

Re: postgresql + ambari server failed to open port 8080

Super Mentor

@mike_bronson7 

As mentioned earlier the PATH "/usr/lib/ambari-server/postgresql-42.2.2.jar" might be different in your casse.

 

So please check what is the JAR file name in your case.

# ls -lart /usr/lib/ambari-server/postgresql-*

Re: postgresql + ambari server failed to open port 8080

about the second test

 

how to find the HOSTNAME of PostgresDB ?

Michael-Bronson

Re: postgresql + ambari server failed to open port 8080

Super Mentor

@mike_bronson7 

As you are using embedded Local postgres instance hence you can replace "localhost" with `hostname -f` like following. Or just use the output of "hostname -f" command in the previously mentioned command to replace "localhost"

 

# /usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-42.2.2.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://`hostname -f`:5432/ambari ambari bigdata org.postgresql.Driver 

 

.

Re: postgresql + ambari server failed to open port 8080

 

 

now I see the correct syntax ( I use full FQDN )

 

 

/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-9.3-1101-jdbc4.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://`hostname -f`:5432/ambari ambari bigdata org.postgresql.Driver
Connected to DB Successfully!

Michael-Bronson

Re: postgresql + ambari server failed to open port 8080

sorry for my mistake

 

 

the correct is

 

/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-9.3-1101-jdbc4.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://localhost:5432/ambari ambari bigdata org.postgresql.Driver

 

 

but its hang ( not return prompt ) 

Michael-Bronson

Re: postgresql + ambari server failed to open port 8080

Super Mentor

@mike_bronson7 

As you mentioned test Db command  " its hang ( not return prompt ) " This indicates some issue from Postgres side.

 

Hence can you please try restarting Postgres DB once . then re run the Test command again.

 

Still if you face any issue then please share the full Postgres log.

 

Re: postgresql + ambari server failed to open port 8080

now I see the correct syntax ( I use full FQDN )

 

 

/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-9.3-1101-jdbc4.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://`hostname -f`:5432/ambari ambari bigdata org.postgresql.Driver
Connected to DB Successfully!

Michael-Bronson