Ambari-agent registration failed

[#1]

I am trying to register ambari agent on my datanodes vai the ambari server web interface and getting below error while starting the ambari agent script. I have setup a local yum repository AMBARI-2.2.1.0 and HDP-2.4.0.0. I was successfully able to install/start the ambari server but while registering the ambari agent i am facing error - "Ambari Agent host cannot reach Ambari Server 'ns2.dns2go.com:8080'". I am not sure where is ns2.dns2go.com coming into picture.

==========================

Creating target directory...
==========================

Command start time 2017-04-05 12:33:58

Connection to www.hdpdn1.rajlab.com closed.
SSH command execution finished
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:33:58

==========================
Copying common functions script...
==========================

Command start time 2017-04-05 12:33:58

scp /usr/lib/python2.6/site-packages/ambari_commons
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:33:58

==========================
Copying OS type check script...
==========================

Command start time 2017-04-05 12:33:58

scp /usr/lib/python2.6/site-packages/ambari_server/os_check_type.py
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:33:58

==========================
Running OS type check...
==========================

Command start time 2017-04-05 12:33:58
Cluster primary/cluster OS family is redhat6 and local/current OS family is redhat6

Connection to www.hdpdn1.rajlab.com closed.
SSH command execution finished
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:33:59

==========================
Checking 'sudo' package on remote host...
==========================

Command start time 2017-04-05 12:33:59
sudo-1.8.6p3-15.el6.x86_64

Connection to www.hdpdn1.rajlab.com closed.
SSH command execution finished
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:33:59

==========================
Copying repo file to 'tmp' folder...
==========================

Command start time 2017-04-05 12:33:59

scp /etc/yum.repos.d/ambari.repo
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:33:59

==========================
Moving file to repo dir...
==========================

Command start time 2017-04-05 12:33:59

Connection to www.hdpdn1.rajlab.com closed.
SSH command execution finished
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:33:59

==========================
Changing permissions for ambari.repo...
==========================

Command start time 2017-04-05 12:33:59

Connection to www.hdpdn1.rajlab.com closed.
SSH command execution finished
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:34:00

==========================
Copying setup script file...
==========================

Command start time 2017-04-05 12:34:00

scp /usr/lib/python2.6/site-packages/ambari_server/setupAgent.py
host=www.hdpdn1.rajlab.com, exitcode=0
Command end time 2017-04-05 12:34:00

==========================
Running setup agent script...
==========================

Command start time 2017-04-05 12:34:00
Host registration aborted. Ambari Agent host cannot reach Ambari Server 'ns2.dns2go.com:8080'. Please check the network connectivity between the Ambari Agent host and the Ambari Server

Connection to www.hdpdn1.rajlab.com closed.
SSH command execution finished
host=www.hdpdn1.rajlab.com, exitcode=1
Command end time 2017-04-05 12:35:03

ERROR: Bootstrap of host www.hdpdn1.rajlab.com fails because previous action finished with non-zero exit code (1)
ERROR MESSAGE: tcgetattr: Invalid argument
Connection to www.hdpdn1.rajlab.com closed.

STDOUT: Host registration aborted. Ambari Agent host cannot reach Ambari Server 'ns2.dns2go.com:8080'. Please check the network connectivity between the Ambari Agent host and the Ambari Server

Connection to www.hdpdn1.rajlab.com closed.

also when i checked the ambari-server logs i could see some bootstrap mesgs.

[root@hdpserver yum.repos.d]# tail -50 /var/log/ambari-server/ambari-server.log STDOUT: Host registration aborted. Ambari Agent host cannot reach Ambari Server 'ns2.dns2go.com:8080'. Please check the network connectivity between the Ambari Agent host and the Ambari Server tcgetattr: Invalid argument Connection to www.hdpdn1.rajlab.com closed. ERROR:root:ERROR: Bootstrap of host www.hdpdn3.rajlab.com fails because previous action finished with non-zero exit code (1) ERROR MESSAGE: tcgetattr: Invalid argument Connection to www.hdpdn3.rajlab.com closed. STDOUT: Host registration aborted. Ambari Agent host cannot reach Ambari Server 'ns2.dns2go.com:8080'. Please check the network connectivity between the Ambari Agent host and the Ambari Server tcgetattr: Invalid argument Connection to www.hdpdn3.rajlab.com closed. ERROR:root:ERROR: Bootstrap of host www.hdpdn2.rajlab.com fails because previous action finished with non-zero exit code (1) ERROR MESSAGE: tcgetattr: Invalid argument Connection to www.hdpdn2.rajlab.com closed. STDOUT: Host registration aborted. Ambari Agent host cannot reach Ambari Server 'ns2.dns2go.com:8080'. Please check the network connectivity between the Ambari Agent host and the Ambari Server tcgetattr: Invalid argument Connection to www.hdpdn2.rajlab.com closed. INFO:root:Finished parallel bootstrap 05 Apr 2017 12:35:05,396 INFO [pool-11-thread-1] BSHostStatusCollector:55 - Request directory /var/run/ambari-server/bootstrap/4

Could someone pease guide me how to proceed further. I have tried to install manually the ambari-agent as well on each node but still i am not able to register the host from ambari server interface. Kindly help

[#2 · Accepted solution]

@Rahul Jain

In your error stack i see the following entry:

Host registration aborted. Ambari Agent host cannot reach Ambari Server 'ns2.dns2go.com:8080'. Please check the network connectivity between the Ambari Agent host and the Ambari Server

.

Can you please make sure that the hostnames are correctly used and every agent host machine and ambari host machine is returning the correct hostname (FQDN) in the following commands output?

# hostname -f

.

From the ambari agent machine try to access the ambari server's 8080 port to see if there is no firewall restriction.

# telnet  ns2.dns2go.com  8080

- If you are not able to connect then there may be a firewall or Network issue between the ambari server & agent machine. If IP Tables is enabled on ambari then try disabling it.

- Make sue that the "/etc/hosts" file of ambari agent hosts as well as ambari server has the entries of all the cluster members in it.

Example: following kind of entries exist on every cluster member machine so that they can resolve each other.

10.10.20.120	ns2.dns2go.com
10.10.20.130	agent1.dns2go.com
10.10.20.140	agent2.dns2go.com
10.10.20.150	agent3.dns2go.com

.

[#3]

@Rahul Jain

Additionally as you mentioned that you tried installing ambari agents manually so please check the "/etc/ambari-agent/conf/ambari-agent.ini" file to see if you have entered correct port information's there:

[server]
hostname=sandbox.hortonworks.com
url_port=8440
secured_url_port=8441

[agent]
logdir=/var/log/ambari-agent
piddir=/var/run/ambari-agent
prefix=/var/lib/ambari-agent/data
;loglevel=(DEBUG/INFO)
loglevel=INFO
data_cleanup_interval=86400
data_cleanup_max_age=2592000
data_cleanup_max_size_MB = 100
ping_port=8670

.

For Example the default ping port should be "ping_port=8670" , "url_port=8440", "secured_url_port=8441" Ambari agents communicates to the ambari server using these ports. So make sure that these ports are not blocked.

-> https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-reference/content/default_network...

.

[#4]

Hi Jey,

Thanks for your response. The corrected the server hostname and prefixed it with www. and it worked fine. Thanks for your help.

Thanks

Rahul

[#5]

@Rahul Jain

Great to hear that your issue is resolved as per suggestion. It will be very helpful for other as well if you can mark this HCC thread as "Accepted" (Answered).