Installation failed. Unable to receive a heartbeat from the agent.

[#1]

I have four machines, one of which is installed with CM server and cm agent, but this machine always reports that the installation fails when checking the web. Unable to receive a heartbeat from the agent. I have checked it through the prompt. There is no problem. It is the same as the other three. The following is the startup log of the agent

[23/Jun/2022 14:17:05 +0000] 4681 MainThread agent INFO Stopping agent...
[23/Jun/2022 14:17:05 +0000] 4681 MainThread agent INFO No extant cgroups; unmounting any cgroup roots
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPING
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) shut down
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Stopped thread '_TimeoutMonitor'.
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPED
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPING
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) already shut down
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE No thread running for None.
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPED
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus EXITING
[23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus EXITED
[23/Jun/2022 14:17:06 +0000] 4681 MainThread agent INFO Agent exiting; caught signal 15
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO ================================================================================
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO SCM Agent Version: 6.3.1
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent Protocol Version: 4
[23/Jun/2022 14:17:07 +0000] 5352 MainThread __init__ INFO Agent UUID file was last modified at 2022-06-22 15:28:23.208587
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Using Host ID: 788cfe56-58f9-4c12-a5a1-53500bd6bf9f
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Using directory: /run/cloudera-scm-agent
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Using supervisor binary path: /opt/cloudera/cm-agent/bin/../bin/supervisord
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent Logging Level: INFO
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent config:
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Security.use_tls = 0
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Security.max_cert_depth = 9
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.orphan_process_dir_staleness_threshold = 5184000
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.dns_resolution_collection_timeout_seconds = 30
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.orphan_process_dir_refresh_interval = 3600
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.local_filesystem_whitelist = ext2,ext3,ext4,xfs
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.stacks_log_bundle_max_bytes = 1073741824
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.metrics_url_timeout_seconds = 30.0
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.max_collection_wait_seconds = 10.0
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.task_metrics_timeout_seconds = 5.0
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.stacks_log_max_uncompressed_file_size_bytes = 5242880
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.scm_debug = INFO
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.server_host = 172.20.1.140
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.monitored_nodev_filesystem_types = nfs,nfs4,tmpfs
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.listening_hostname = cdh-manager
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.server_port = 7182
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.dns_resolution_collection_interval_seconds = 60
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.impala_profile_bundle_max_bytes = 1073741824
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Neither verify_cert_file nor verify_cert_dir are configured. Not performing validation of server certificates in HTTPS communication. These options can be configured in this agent's config.ini file to enable certificate validation.
[23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent starting as pid 5352 user root(0) group root(0).
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem blkio at /sys/fs/cgroup/blkio
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem devices at /sys/fs/cgroup/devices
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem cpuacct at /sys/fs/cgroup/cpu,cpuacct
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem cpu at /sys/fs/cgroup/cpu,cpuacct
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem memory at /sys/fs/cgroup/memory
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: cpu
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'writable_cgroup_dot_procs': True, 'default_cpu_shares': 1024, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True}
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: cpuacct
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_cpuacct': True}
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: memory
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_memory': True, 'default_memory_limit_in_bytes': 8796093022207, 'default_memory_soft_limit_in_bytes': 8796093022207}
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: devices
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_devices': True}
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: blkio
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_blkio': True, 'default_blkio_weight': 1000}
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_memory_limit_in_bytes
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_blkio_weight
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: writable_cgroup_dot_procs
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_cpu_rt_runtime_us
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_memory_soft_limit_in_bytes
[23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_cpu_shares
[23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Found cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes': 8796093022207, 'default_blkio_weight': 1000, 'writable_cgroup_dot_procs': True, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True, 'default_memory_soft_limit_in_bytes': 8796093022207, 'has_cpuacct': True, 'default_cpu_shares': 1024, 'has_devices': True, 'has_blkio': True}
[23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
[23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Using timeout of 2.000000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Using join timeout of 0.100000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Using tolerance of 60.000000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs']
[23/Jun/2022 14:17:09 +0000] 5352 MainThread kt_renewer INFO Agent wide credential cache set to /var/run/cloudera-scm-agent/krb5cc_cm_agent_0
[23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Setting up supervisord event monitor.
[23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Using metrics_url_timeout_seconds of 30.000000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Using max_collection_wait_seconds of 10.000000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread metrics INFO Importing tasktracker metric schema from file /opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/tasktracker/schema.json
[23/Jun/2022 14:17:09 +0000] 5352 MainThread ntp_monitor INFO Using timeout of 2.000000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread dns_names INFO Using timeout of 30.000000
[23/Jun/2022 14:17:09 +0000] 5352 MainThread __init__ INFO Created DNS monitor.
[23/Jun/2022 14:17:09 +0000] 5352 MainThread stacks_collection_manager INFO Using max_uncompressed_file_size_bytes: 5242880
[23/Jun/2022 14:17:09 +0000] 5352 MainThread daemon INFO Importing metric schema from file /opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/schema.json
[23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CMF_PACKAGE_DIR': '/opt/cloudera/cm-agent/service', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUSTEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/opt/cloudera/cm/lib/postgresql-42.1.4.jre7.jar', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_PARQUET_HOME': '/usr/lib/parquet', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/opt/cloudera/cm', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_SERVER_ROOT': '/opt/cloudera/cm', 'CDH_KUDU_HOME': '/usr/lib/kudu', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_HUE_HOME': '/usr/lib/hue', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
[23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
[23/Jun/2022 14:17:11 +0000] 5352 MainThread supervisor INFO Trying to connect to supervisor (Attempt 1)
[23/Jun/2022 14:17:11 +0000] 5352 MainThread supervisor INFO Supervisor version: 3.0, pid: 1129
[23/Jun/2022 14:17:11 +0000] 5352 MainThread supervisor INFO Successfully connected to supervisor
[23/Jun/2022 14:17:11 +0000] 5352 MainThread agent INFO Supervisor version: 3.0, pid: 1129
[23/Jun/2022 14:17:11 +0000] 5352 MainThread agent INFO Connecting to previous supervisor: agent-1129-1655964031.
[23/Jun/2022 14:17:13 +0000] 5352 MainThread supervisor INFO Triggering supervisord update.
[23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Bus STARTING
[23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Started monitor thread '_TimeoutMonitor'.
[23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Serving on http://127.0.0.1:9001
[23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Bus STARTED
[23/Jun/2022 14:17:15 +0000] 5352 MainThread daemon INFO New monitor: (<cmf.monitor.host.HostMonitor object at 0x7f3495dc4450>,)
[23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Setting default socket timeout to 45
[23/Jun/2022 14:17:15 +0000] 5352 MonitorDaemon-Scheduler daemon INFO Monitor ready to report: ('HostMonitor',)
[23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Previously active parcels: {}
[23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Loading last saved hb response to complete initialization: /var/lib/cloudera-scm-agent/response.avro
[23/Jun/2022 14:17:15 +0000] 5352 Monitor-HostMonitor network_interfaces INFO NIC iface eth0 doesn't support ETHTOOL (95)
[23/Jun/2022 14:17:15 +0000] 5352 MainThread heartbeat_tracker INFO HB stats (seconds): num:1 LIFE_MIN:0.02 min:0.02 mean:0.02 max:0.02 LIFE_MAX:0.02
[23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO CM server guid: 78d0ba5f-6d53-4812-af67-6607fda2a71b
[23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels
[23/Jun/2022 14:17:15 +0000] 5352 MainThread parcel INFO Agent does create users/groups and apply file permissions
[23/Jun/2022 14:17:15 +0000] 5352 MainThread downloader INFO Downloader path: /opt/cloudera/parcel-cache
[23/Jun/2022 14:17:15 +0000] 5352 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache
[23/Jun/2022 14:17:15 +0000] 5352 MainThread throttling_logger WARNING Failed parsing alternatives line: pax string index out of range link currently points to /usr/bin/spax
[23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Flood daemon (re)start attempt
[23/Jun/2022 14:17:17 +0000] 5352 MainThread agent INFO Reloading status server config, sending SIGHUP
[23/Jun/2022 14:17:17 +0000] 5352 MainThread throttling_logger INFO Identified java component java8 with full version java version "1.8.0_301" Java(TM) SE Runtime Environment (build 1.8.0_301-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.301-b09, mixed mode) for requested version .
[23/Jun/2022 14:17:45 +0000] 5352 DnsResolutionMonitor throttling_logger WARNING hostname cdh-manager differs from the canonical name iZuf60e0xat656rqmz890jZ

[#2]

@rookie-xxd ,

Could you provide the following information:

• Are you using TLS?
• A screenshot of the "Hosts" page in Cloudera Manager showing the registered hosts
• The content of the /etc/hosts file on the machine that is failing to heartbeat

Cheers,

André