Created 06-15-2017 10:32 PM
As the subject line goes, I'm confused by whether if Chrony or NTPD is commonly used for installing cloudera.
Created 06-16-2017 12:14 AM
Hello DDreamer94!
CDH requires that you configure the NTP service on each machine in your cluster. For more information, please take a look into:
https://www.cloudera.com/documentation/enterprise/latest/topics/install_cdh_enable_ntp.html
Thanks,
Laith
Created 06-16-2017 11:15 AM
NTP or Chrony should work.
From the Clock Offset Host Health check (CM 5.8.2).
"This is a host health test that checks if the host's system clock appears to be out-of-sync with its NTP server(s). The test uses the 'ntpdc -np' (if ntpd is running) or 'chronyc sources' (if chronyd is running) command to check that the host is synchronized to an NTP peer and that the absolute value of the host's clock offset from that peer is not too large. If the command fails, NTP is not synchronized to a server, or the host's NTP daemon is not running or cannot be contacted, the test will return "Bad" health.
The 'ntpdc -np' or 'chronyc sources' output contains a row for each of the host's NTP servers. The row starting with a '*' (if ntpdc) or '^*' (if chronyc) contains the peer to which the host is currently synchronized. No row starting with a '*' or '^*' indicates that the host is not currently synchronized. Communication errors and too large an offset between the peer and the host time are examples of conditions that can lead to a host being unsynchronized.
Make sure that UDP port 123 is open in any firewall that is in use. Check the system log for ntpd or chronyd messages related to configuration errors. If running ntpd, use 'ntpdc -c iostat' to verify that packets are sent and recieved between the different peers. More information about the conditions of each peer can be found by running the command 'ntpq -c as'. The output of this command includes the association ID that can be used in combination with 'ntpq -c "rv "' to get more information about the status of each such peer. The command 'ntpq -c pe' can also be used to return a summary of all peers and the reason why they are not in use. If running chronyd, use 'chronyc activity' to check how many NTP sources are online/offline. More information about the conditions of each peer can be found by running the command 'chronyc sourcestats'. To check chrony tracking, issue the command 'chronyc tracking'.
If NTP is not in use on the host, this check should be disabled for the host using the configuration options shown below. Cloudera recommends using NTP for time synchronization of Hadoop clusters.
A failure of this health test can indicate a problem with the host's NTP service or configuration."
Created 06-16-2017 12:14 AM
Hello DDreamer94!
CDH requires that you configure the NTP service on each machine in your cluster. For more information, please take a look into:
https://www.cloudera.com/documentation/enterprise/latest/topics/install_cdh_enable_ntp.html
Thanks,
Laith
Created 06-16-2017 11:12 AM
Created 06-16-2017 11:15 AM
NTP or Chrony should work.
From the Clock Offset Host Health check (CM 5.8.2).
"This is a host health test that checks if the host's system clock appears to be out-of-sync with its NTP server(s). The test uses the 'ntpdc -np' (if ntpd is running) or 'chronyc sources' (if chronyd is running) command to check that the host is synchronized to an NTP peer and that the absolute value of the host's clock offset from that peer is not too large. If the command fails, NTP is not synchronized to a server, or the host's NTP daemon is not running or cannot be contacted, the test will return "Bad" health.
The 'ntpdc -np' or 'chronyc sources' output contains a row for each of the host's NTP servers. The row starting with a '*' (if ntpdc) or '^*' (if chronyc) contains the peer to which the host is currently synchronized. No row starting with a '*' or '^*' indicates that the host is not currently synchronized. Communication errors and too large an offset between the peer and the host time are examples of conditions that can lead to a host being unsynchronized.
Make sure that UDP port 123 is open in any firewall that is in use. Check the system log for ntpd or chronyd messages related to configuration errors. If running ntpd, use 'ntpdc -c iostat' to verify that packets are sent and recieved between the different peers. More information about the conditions of each peer can be found by running the command 'ntpq -c as'. The output of this command includes the association ID that can be used in combination with 'ntpq -c "rv "' to get more information about the status of each such peer. The command 'ntpq -c pe' can also be used to return a summary of all peers and the reason why they are not in use. If running chronyd, use 'chronyc activity' to check how many NTP sources are online/offline. More information about the conditions of each peer can be found by running the command 'chronyc sourcestats'. To check chrony tracking, issue the command 'chronyc tracking'.
If NTP is not in use on the host, this check should be disabled for the host using the configuration options shown below. Cloudera recommends using NTP for time synchronization of Hadoop clusters.
A failure of this health test can indicate a problem with the host's NTP service or configuration."
Created 06-19-2017 01:16 AM
Thanks for the input, I've actually gone through the documentation, and was setting up the CentOS using the step suggested. I've just cleared this doubt.
Created 06-19-2017 01:22 AM
Hello DDreaner94,
Thanks for confirning that the docunmation works for you, and I agree with @mbigelow, CDH itself isn't really doing the checking so much as relying on system time. The NTP check and Chrony check are doing by the Agent since there are some time sensitive components in the stack.
Please let us know if you have any other concers..
Have a nice day!
Laith
Created on 06-19-2017 01:36 AM - edited 06-19-2017 01:36 AM
@Laith,
The help was much appreciated.
Regards,
DDreamer94
Created 06-19-2017 10:43 AM
For a long time, I was using ntp only. In the last project, I had to use Chrony and since that, I will never back to ntp 🙂
I recommend Chrony.
Created 06-19-2017 11:06 AM
Created 05-21-2019 10:15 PM
I have a problem in one host in my cluster. Clock offset issue. It gives an error like this :
May/2019 21:57:44 +0000] 26424 Metadata-Plugin navigator_plugin INFO Refreshing Metadata Plugin for keytrustee-KMS_KEYTRUSTEE with pipelines []
[21/May/2019 21:57:44 +0000] 26424 Audit-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for keytrustee-KMS_KEYTRUSTEE (log dir: None)
[21/May/2019 21:57:44 +0000] 26424 Metadata-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for keytrustee-KMS_KEYTRUSTEE (log dir: None)
[21/May/2019 21:58:35 +0000] 26424 Monitor-HostMonitor throttling_logger ERROR chronyc: chronyc sources: not synchronized to any server
All the other hosts works fine but only this host doesn't work becaus of clock offset.
1. it is pointing to the right NTP server and is sycnhronised too. chronyd is disabled like in all other hosts. ntpd is active and running.
Any help is appreciated on why cloudera agent is checking for chronyd on this one host?