Support Questions

Find answers, ask questions, and share your expertise

Ambari Confirm hosts failed

avatar
Contributor
Failed to connect to https://centos7-node1:8440/ca due to [Errno 111] \xe6\x8b\x92\xe7\xbb\x9d\xe8\xbf\x9e\xe6\x8e\xa5 

Help!

Thank you!

Creating target directory...
==========================

Command start time 2017-05-05 06:04:58

Connection to centos7-node1 closed.
SSH command execution finished
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:04:58

==========================
Copying ambari sudo script...
==========================

Command start time 2017-05-05 06:04:58

scp /var/lib/ambari-server/ambari-sudo.sh
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:04:59

==========================
Copying common functions script...
==========================

Command start time 2017-05-05 06:04:59

scp /usr/lib/python2.6/site-packages/ambari_commons
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:04:59

==========================
Copying create-python-wrap script...
==========================

Command start time 2017-05-05 06:04:59

scp /var/lib/ambari-server/create-python-wrap.sh
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:05:00

==========================
Copying OS type check script...
==========================

Command start time 2017-05-05 06:05:00

scp /usr/lib/python2.6/site-packages/ambari_server/os_check_type.py
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:05:01

==========================
Running create-python-wrap script...
==========================

Command start time 2017-05-05 06:05:01

Connection to centos7-node1 closed.
SSH command execution finished
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:05:01

==========================
Running OS type check...
==========================

Command start time 2017-05-05 06:05:01
Cluster primary/cluster OS family is redhat7 and local/current OS family is redhat7

Connection to centos7-node1 closed.
SSH command execution finished
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:05:02

==========================
Checking 'sudo' package on remote host...
==========================

Command start time 2017-05-05 06:05:02

Connection to centos7-node1 closed.
SSH command execution finished
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:05:02

==========================
Copying required files...
Ambari repo file not found: /etc/yum.repos.d/ambari.repo
==========================
Copying setup script file...
==========================

Command start time 2017-05-05 06:05:02

scp /usr/lib/python2.6/site-packages/ambari_server/setupAgent.py
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:05:03

==========================
Running setup agent script...
==========================

Command start time 2017-05-05 06:05:03
Repository base is listed more than once in the configuration
Repository updates is listed more than once in the configuration
Repository extras is listed more than once in the configuration
Repository centosplus is listed more than once in the configuration
('INFO 2017-05-05 13:45:19,163 Controller.py:495 - Finished heartbeating and registering cycle
INFO 2017-05-05 13:45:19,163 Controller.py:501 - Controller thread has successfully finished
INFO 2017-05-05 13:45:20,749 ExitHelper.py:56 - Performing cleanup before exiting...
INFO 2017-05-05 13:45:20,749 threadpool.py:111 - Shutting down thread pool
INFO 2017-05-05 13:45:20,750 scheduler.py:606 - Scheduler has been shut down
INFO 2017-05-05 13:45:20,750 threadpool.py:52 - Started thread pool with 3 core threads and 20 maximum threads
INFO 2017-05-05 13:45:20,750 AlertSchedulerHandler.py:168 - [AlertScheduler] Stopped the alert scheduler.
INFO 2017-05-05 13:45:20,750 threadpool.py:111 - Shutting down thread pool
INFO 2017-05-05 13:45:20,750 ExitHelper.py:70 - Cleanup finished, exiting with code:0
INFO 2017-05-05 14:05:11,105 main.py:143 - loglevel=logging.INFO
INFO 2017-05-05 14:05:11,105 main.py:143 - loglevel=logging.INFO
INFO 2017-05-05 14:05:11,105 main.py:143 - loglevel=logging.INFO
INFO 2017-05-05 14:05:11,106 DataCleaner.py:39 - Data cleanup thread started
INFO 2017-05-05 14:05:11,107 DataCleaner.py:120 - Data cleanup started
INFO 2017-05-05 14:05:11,108 DataCleaner.py:122 - Data cleanup finished
INFO 2017-05-05 14:05:11,154 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2017-05-05 14:05:11,156 main.py:430 - Connecting to Ambari server at https://centos7-node1:8440 (192.168.126.128)
INFO 2017-05-05 14:05:11,156 NetUtil.py:67 - Connecting to https://centos7-node1:8440/ca
WARNING 2017-05-05 14:05:11,157 NetUtil.py:98 - Failed to connect to https://centos7-node1:8440/ca due to [Errno 111] \xe6\x8b\x92\xe7\xbb\x9d\xe8\xbf\x9e\xe6\x8e\xa5  
WARNING 2017-05-05 14:05:11,157 NetUtil.py:121 - Server at https://centos7-node1:8440 is not reachable, sleeping for 10 seconds...
', None)
('INFO 2017-05-05 13:45:19,163 Controller.py:495 - Finished heartbeating and registering cycle
INFO 2017-05-05 13:45:19,163 Controller.py:501 - Controller thread has successfully finished
INFO 2017-05-05 13:45:20,749 ExitHelper.py:56 - Performing cleanup before exiting...
INFO 2017-05-05 13:45:20,749 threadpool.py:111 - Shutting down thread pool
INFO 2017-05-05 13:45:20,750 scheduler.py:606 - Scheduler has been shut down
INFO 2017-05-05 13:45:20,750 threadpool.py:52 - Started thread pool with 3 core threads and 20 maximum threads
INFO 2017-05-05 13:45:20,750 AlertSchedulerHandler.py:168 - [AlertScheduler] Stopped the alert scheduler.
INFO 2017-05-05 13:45:20,750 threadpool.py:111 - Shutting down thread pool
INFO 2017-05-05 13:45:20,750 ExitHelper.py:70 - Cleanup finished, exiting with code:0
INFO 2017-05-05 14:05:11,105 main.py:143 - loglevel=logging.INFO
INFO 2017-05-05 14:05:11,105 main.py:143 - loglevel=logging.INFO
INFO 2017-05-05 14:05:11,105 main.py:143 - loglevel=logging.INFO
INFO 2017-05-05 14:05:11,106 DataCleaner.py:39 - Data cleanup thread started
INFO 2017-05-05 14:05:11,107 DataCleaner.py:120 - Data cleanup started
INFO 2017-05-05 14:05:11,108 DataCleaner.py:122 - Data cleanup finished
INFO 2017-05-05 14:05:11,154 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2017-05-05 14:05:11,156 main.py:430 - Connecting to Ambari server at https://centos7-node1:8440 (192.168.126.128)
INFO 2017-05-05 14:05:11,156 NetUtil.py:67 - Connecting to https://centos7-node1:8440/ca
WARNING 2017-05-05 14:05:11,157 NetUtil.py:98 - Failed to connect to https://centos7-node1:8440/ca due to [Errno 111] \xe6\x8b\x92\xe7\xbb\x9d\xe8\xbf\x9e\xe6\x8e\xa5  
WARNING 2017-05-05 14:05:11,157 NetUtil.py:121 - Server at https://centos7-node1:8440 is not reachable, sleeping for 10 seconds...
', None)

Connection to centos7-node1 closed.
SSH command execution finished
host=centos7-node1, exitcode=0
Command end time 2017-05-05 06:05:13

Registering with the server...
Registration with the server failed.
1 ACCEPTED SOLUTION

avatar
Master Mentor

@frank chen

From your logs we see that agents are failing to communicate with ambari-server.

INFO 2017-05-05 14:05:11,156 NetUtil.py:67 - Connecting to https://centos7-node1:8440/ca
WARNING 2017-05-05 14:05:11,157 NetUtil.py:98 - Failed to connect to https://centos7-node1:8440/ca due to [Errno 111] \xe6\x8b\x92\xe7\xbb\x9d\xe8\xbf\x9e\xe6\x8e\xa5  
WARNING 2017-05-05 14:05:11,157 NetUtil.py:121 - Server at https://centos7-node1:8440 is not reachable, sleeping for 10 seconds...

Please make sure that the agents are able to access the ambari-server hostname & port properly. There should be no iptables/firewall restriction.

Please check that you are able to do telnet from agent machine to connect to ambari server?

8440: Handshake Port for Ambari Agents to Ambari Server

8441: Registration and Heartbeat Port for Ambari Agents to Ambari Server

  # telnet centos7-node1 8440
  # telnet centos7-node1 8441
  # telnet centos7-node1 8080

Also on ambari server and agent hosts please make sure that the hostnames are resolvable throughout the cluster (means every host has the correct "/etc/hosts" entry to resolve other hosts.

The output of the following command shoudl be resolvable and consistent through out the cluster hosts.

# hostname -f

https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/set_the_host...

.

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@frank chen

From your logs we see that agents are failing to communicate with ambari-server.

INFO 2017-05-05 14:05:11,156 NetUtil.py:67 - Connecting to https://centos7-node1:8440/ca
WARNING 2017-05-05 14:05:11,157 NetUtil.py:98 - Failed to connect to https://centos7-node1:8440/ca due to [Errno 111] \xe6\x8b\x92\xe7\xbb\x9d\xe8\xbf\x9e\xe6\x8e\xa5  
WARNING 2017-05-05 14:05:11,157 NetUtil.py:121 - Server at https://centos7-node1:8440 is not reachable, sleeping for 10 seconds...

Please make sure that the agents are able to access the ambari-server hostname & port properly. There should be no iptables/firewall restriction.

Please check that you are able to do telnet from agent machine to connect to ambari server?

8440: Handshake Port for Ambari Agents to Ambari Server

8441: Registration and Heartbeat Port for Ambari Agents to Ambari Server

  # telnet centos7-node1 8440
  # telnet centos7-node1 8441
  # telnet centos7-node1 8080

Also on ambari server and agent hosts please make sure that the hostnames are resolvable throughout the cluster (means every host has the correct "/etc/hosts" entry to resolve other hosts.

The output of the following command shoudl be resolvable and consistent through out the cluster hosts.

# hostname -f

https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/set_the_host...

.

avatar
Contributor

Yes,you are right!

Thank you!

avatar
Master Mentor

@frank chen

If this resolves your issue then please click on the "Accept" button and lark this thread as Answered. So that will be useful for other users as well.