Support Questions

cjervis · ‎08-24-2020

BEGIN sudo systemctl restart cloudera-scm-agent
END (0)
agent logs:
BEGIN sudo tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/'
tail: cannot open '/var/log/cloudera-scm-agent//cloudera-scm-agent.out' for reading: No such file or directory
tail: cannot open '/var/log/cloudera-scm-agent//cloudera-scm-agent.out' for reading: No such file or directory
END (0)
BEGIN sudo tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/'
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Bus STARTING
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Started monitor thread '_TimeoutMonitor'.
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Serving on http://127.0.0.1:9001
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Bus STARTED
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread status_server INFO Status server url is http://ip-10-13-35-219:9000/
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread daemon INFO New monitor: (<cmf.monitor.host.HostMonitor object at 0x7fb2308fac90>,)
>>[24/Aug/2020 07:42:48 +0000] 18092 MonitorDaemon-Scheduler daemon INFO Monitor ready to report: ('HostMonitor',)
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Setting default socket timeout to 45
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Previously active parcels: {}
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Loading last saved hb response to complete initialization: /var/lib/cloudera-scm-agent/response.avro
>>[24/Aug/2020 07:42:48 +0000] 18092 Monitor-HostMonitor network_interfaces INFO NIC iface eth0 doesn't support ETHTOOL (95)
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread heartbeat_tracker INFO HB stats (seconds): num:1 LIFE_MIN:0.02 min:0.02 mean:0.02 max:0.02 LIFE_MAX:0.02
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO CM server guid: 02d62a02-5027-4a62-9726-7c72df9b1882
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread parcel INFO Agent does create users/groups and apply file permissions
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread downloader INFO Downloader path: /opt/cloudera/parcel-cache
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread throttling_logger WARNING Failed parsing alternatives line: nc string index out of range link best version is /bin/nc.openbsd
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Flood daemon (re)start attempt
>>[24/Aug/2020 07:42:49 +0000] 18092 MainThread agent INFO Reloading status server config, sending SIGHUP
>>[24/Aug/2020 07:42:49 +0000] 18092 MainThread throttling_logger INFO Identified java component java8 with full version openjdk version "1.8.0_265" OpenJDK Runtime Environment (build 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01) OpenJDK 64-Bit Server VM (build 25.265-b01, mixed mode) for requested version .
>>[24/Aug/2020 07:43:18 +0000] 18092 DnsResolutionMonitor throttling_logger WARNING hostname ip-10-13-35-219 differs from the canonical name 10.13.35.219
>>[24/Aug/2020 07:45:13 +0000] 18092 Monitor-HostMonitor throttling_logger ERROR Failed to collect NTP metrics
>>Traceback (most recent call last):
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/host/ntp_monitor.py", line 46, in collect
>> self.collect_chronyd()
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/host/ntp_monitor.py", line 56, in collect_chronyd
>> result, stdout, stderr = self._subprocess_with_timeout(args, self._timeout)
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/host/ntp_monitor.py", line 39, in _subprocess_with_timeout
>> return subprocess_with_timeout(args, timeout)
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/subprocess_timeout.py", line 50, in subprocess_with_timeout
>> p = subprocess.Popen(**kwargs)
>> File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
>> errread, errwrite)
>> File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
>> raise child_exception
>>OSError: [Errn>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Bus STARTING
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Started monitor thread '_TimeoutMonitor'.
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Serving on http://127.0.0.1:9001
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:42:46] ENGINE Bus STARTED
>>[24/Aug/2020 07:42:46 +0000] 18092 MainThread status_server INFO Status server url is http://ip-10-13-35-219:9000/
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread daemon INFO New monitor: (<cmf.monitor.host.HostMonitor object at 0x7fb2308fac90>,)
>>[24/Aug/2020 07:42:48 +0000] 18092 MonitorDaemon-Scheduler daemon INFO Monitor ready to report: ('HostMonitor',)
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Setting default socket timeout to 45
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Previously active parcels: {}
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Loading last saved hb response to complete initialization: /var/lib/cloudera-scm-agent/response.avro
>>[24/Aug/2020 07:42:48 +0000] 18092 Monitor-HostMonitor network_interfaces INFO NIC iface eth0 doesn't support ETHTOOL (95)
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread heartbeat_tracker INFO HB stats (seconds): num:1 LIFE_MIN:0.02 min:0.02 mean:0.02 max:0.02 LIFE_MAX:0.02
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO CM server guid: 02d62a02-5027-4a62-9726-7c72df9b1882
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread parcel INFO Agent does create users/groups and apply file permissions
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread downloader INFO Downloader path: /opt/cloudera/parcel-cache
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread throttling_logger WARNING Failed parsing alternatives line: nc string index out of range link best version is /bin/nc.openbsd
>>[24/Aug/2020 07:42:48 +0000] 18092 MainThread agent INFO Flood daemon (re)start attempt
>>[24/Aug/2020 07:42:49 +0000] 18092 MainThread agent INFO Reloading status server config, sending SIGHUP
>>[24/Aug/2020 07:42:49 +0000] 18092 MainThread throttling_logger INFO Identified java component java8 with full version openjdk version "1.8.0_265" OpenJDK Runtime Environment (build 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01) OpenJDK 64-Bit Server VM (build 25.265-b01, mixed mode) for requested version .
>>[24/Aug/2020 07:43:18 +0000] 18092 DnsResolutionMonitor throttling_logger WARNING hostname ip-10-13-35-219 differs from the canonical name 10.13.35.219
>>[24/Aug/2020 07:45:13 +0000] 18092 Monitor-HostMonitor throttling_logger ERROR Failed to collect NTP metrics
>>Traceback (most recent call last):
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/host/ntp_monitor.py", line 46, in collect
>> self.collect_chronyd()
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/host/ntp_monitor.py", line 56, in collect_chronyd
>> result, stdout, stderr = self._subprocess_with_timeout(args, self._timeout)
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/host/ntp_monitor.py", line 39, in _subprocess_with_timeout
>> return subprocess_with_timeout(args, timeout)
>> File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/subprocess_timeout.py", line 50, in subprocess_with_timeout
>> p = subprocess.Popen(**kwargs)
>> File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
>> errread, errwrite)
>> File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
>> raise child_exception
>>OSError: [Errno 2] No such file or directory
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread agent INFO Stopping agent...
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread agent INFO No extant cgroups; unmounting any cgroup roots
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPING
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) shut down
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Stopped thread '_TimeoutMonitor'.
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPED
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPING
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) already shut down
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE No thread running for None.
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPED
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus EXITING
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus EXITED
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread agent INFO Agent exiting; caught signal 15
o 2] No such file or directory
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread agent INFO Stopping agent...
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread agent INFO No extant cgroups; unmounting any cgroup roots
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPING
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) shut down
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Stopped thread '_TimeoutMonitor'.
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPED
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPING
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) already shut down
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE No thread running for None.
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus STOPPED
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus EXITING
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread _cplogging INFO [24/Aug/2020:07:45:50] ENGINE Bus EXITED
>>[24/Aug/2020 07:45:50 +0000] 18092 MainThread agent INFO Agent exiting; caught signal 15
END (0)

GangWar · ‎08-25-2020

@Prabhsimran Check if chronyd is also running on the server/hosts. You have to disable remove one of them means either you can use NTP or the chronyd. Then restart the clock service and see how it goes.

Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.