unable to start ambari server on EC2 instance during first time setup.

dumpeti_chaitan · ‎12-16-2017

Trying to install and start ambari-server on EC2 ubuntu instance.

Below is the error I am facing:

ubuntu@ec2-34-209-250-154:~$ sudo ambari-server 
start Using python /usr/bin/python 
Starting ambari-server 
Ambari Server running with administrator privileges. 
Organizing resource files at /var/lib/ambari-server/resources... 
Ambari database consistency check started... 
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out 
Server log at: /var/log/ambari-server/ambari-server.log 
Waiting for server start............ERROR: Exiting with exit code -1. 
REASON: Ambari Server java process has stopped. Please check the logs for more information.

Error in /var/log/ambari-server/ambari-server.log:

16 Dec 2017 05:53:27,745  INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.ViewInstanceEventCreator
16 Dec 2017 05:53:27,746  INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.ViewPrivilegeEventCreator
16 Dec 2017 05:53:29,631 ERROR [main] DBAccessorImpl:119 - Error while creating database accessor com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:981)at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:339)at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2254)at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2286)at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2085)at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:795)at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400)

ubuntu@ec2-34-209-250-154:~$ hostname
ec2-34-209-250-154.us-west-2.compute.amazonaws.com
ubuntu@ec2-34-209-250-154:~$ hostname -f
ec2-34-209-250-154.us-west-2.compute.amazonaws.com

For MYSQL port it is showing as below:

ubuntu@ec2-34-209-250-154:~$ netstat -tnlpa | grep 3306
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      -

kindly help me to start ambari-server.

Shelton · ‎12-16-2017

@Chaitanya D

If you get a SQLException: Connection refused or Connection timed out or a MySQL specific CommunicationsException: Communications link failure, then it means that the DB isn't reachable at all. This can have one or more of the following causes:

IP address or hostname in JDBC URL is wrong.
Hostname in JDBC URL is not recognized by local DNS server.
Port number is missing or wrong in JDBC URL.
DB server is down.
DB server doesn't accept TCP/IP connections.
DB server has run out of connections.
Something in between Java and DB is blocking connections, e.g. a firewall or proxy.

To solve the one or the other, follow the following advice:

Verify and test them with ping.
Refresh DNS or use IP address in JDBC URL instead.
Verify it based on my.cnf of MySQL DB.
Start the DB.
Verify if mysqld is started without the --skip-networking option.
Restart the DB and fix your code accordingly that it closes connections in finally.
Disable firewall and/or configure firewall/proxy to allow/forward the port.

Can you especially have a look at the /etc/hosts entries and check the firewall on all the cluster hosts UFW is typically installed by default in Ubuntu.

Disabling UFW

$ sudo ufw disable

Please let me know

dumpeti_chaitan · ‎12-16-2017

@Geoffrey Shelton Okot

Thanks for replying to me, Now the ambari-server is running fine as shown below:

ubuntu@ip-172-31-24-170:~$ sudo ambari-server status
Using python  /usr/bin/python
Ambari-server status
Ambari Server running
Found Ambari Server PID: 1389 at: /var/run/ambari-server/ambari-server.pid

But I could not able to point web browser using any of the below:

http://localhost:8080
http://ip-172-31-24-170.us-west-2.compute.internal:8080
http://ip-172-31-24-170:8080

how can I open ambari web please guide me onthe same.

ubuntu@ip-172-31-24-170:~$ cat /etc/hostname
ip-172-31-24-170
ubuntu@ip-172-31-24-170:~$ cat /etc/hosts
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
ubuntu@ip-172-31-24-170:~$ hostname
ip-172-31-24-170.us-west-2.compute.internal
ubuntu@ip-172-31-24-170:~$ hostname -f
ip-172-31-24-170.us-west-2.compute.internal

ubuntu@ip-172-31-24-170:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 02:37:fe:68:1c:18 
          inet addr:172.31.24.170  Bcast:172.31.31.255  Mask:255.255.240.0
          inet6 addr: fe80::37:feff:fe68:1c18/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
          RX packets:1156 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1036 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:100086 (100.0 KB)  TX bytes:123808 (123.8 KB)
lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:430 errors:0 dropped:0 overruns:0 frame:0
          TX packets:430 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:73193 (73.1 KB)  TX bytes:73193 (73.1 KB)

Could you pls guide me to open ambari web, do I need to configure any thing here?

Thanks in advance.

Shelton · ‎12-16-2017

@Chaitanya D

The hostname/IP you are using is using is (Private DNS )for internal communication within the AWS data center

http://ip-172-31-24-170.us-west-2.compute.internal:8080

If you want to access your Ambari server from your laptop you will need to use the (Public DNS ) like ec2-18-195-169-13.us-west-2.compute.amazonaws.com:8080 please check what matches your Ambari's Public DNS from the AWS console.

Ensure the correct port have been open for external communication in the security group

Please let me know

dumpeti_chaitan · ‎12-16-2017

I tried using public DNS but the login page of ambari web is not accessable. I used as below:

http://ec2-54-245-44-216.us-west-2.compute.amazonaws.com:8080/

also attaching the Inbound and Outbound rules defined in security groups. If there are any changes need to be done pls help me on the same.

Shelton · ‎12-16-2017

@Chaitanya D

You should be able to access your Ambari UI from your laptop, if you are using a windows laptop make sure you have an entry in your local hosts file.

Format

<ip address>  <hostname> [<alias> ...]

Steps

Go to Start > run Notepad.
Right click on the Notepad icon and select Run as a administrator.
Select Open from the File menu option.
Browse to c:\windows\system32\drivers\etc\hosts.
Add the hostname and IP address to the bottom of the host file.
Save the host file.

eg

172.31.28.216    ec2-54-245-44-216.us-west-2.compute.amazonaws.com  myambari

Hope that helps

dumpeti_chaitan · ‎12-16-2017

Hi Geoffrey Shelton Okot,

These are the steps I followed while installing ambari server:

1. created one AWS EC2 instance and connected to that instance using .pem file and public DNS name from my local laptop running with ubuntu os.

2. The ambari server, mysql is running fine on EC2 instance.

3. Now I am trying to connect to ambari web from my local laptop ubuntu with ec2 publi domain name as shown below:

http://ec2-54-218-134-113.us-west-2.compute.amazonaws.com:8080

but it is showing as "This site cant be reached.

4. I can able to ping to ec2 instance public ip:

# ping 54-218-134-113 is working fine.

but # ping ec2-54-218-134-113.us-west-2.compute.amazonaws.com is not working.

so trying with http:54-218-134-113:8080 also but no use.

also I have added below in my local laptop /etc/hosts file:

172.31.24.170 ec2-54-218-134-113.us-west-2.compute.amazonaws.com myambari

also changed my local laptop hostnme to ec2-54-218-134-113.us-west-2.compute.amazonaws.com as shown below:

chaithu@localhost:~$ hostname ec2-54-218-134-113.us-west-2.compute.amazonaws.com

chaithu@localhost:~$ hostname -f ec2-54-218-134-113.us-west-2.compute.amazonaws.com

Is any thing more I need to do that I missed. please suggest me onthe same. Thanks.

Shelton · ‎12-16-2017

@Chaitanya D

Okay now I get the clear picture I was assuming you were on a windows client. Could you add this to your Ubuntu hosts file

<ipaddress> <hostname> [<alias> ...]

54-218-134-113    ec2-54-218-134-113.us-west-2.compute.amazonaws.com         myambari

With the above setup in your hosts file you should be able to access your AWS ambari with

http://ec2-54-218-134-113.us-west-2.compute.amazonaws.com:8080

or the alias (customised) note you can change myambari to anything in your hosts files, my previous input was an example

http://myambari:8080

Please revert