Member since
04-28-2021
1
Post
0
Kudos Received
0
Solutions
06-22-2022
11:21 PM
I have four machines, one of which is installed with CM server and cm agent, but this machine always reports that the installation fails when checking the web. Unable to receive a heartbeat from the agent. I have checked it through the prompt. There is no problem. It is the same as the other three. The following is the startup log of the agent [23/Jun/2022 14:17:05 +0000] 4681 MainThread agent INFO Stopping agent... [23/Jun/2022 14:17:05 +0000] 4681 MainThread agent INFO No extant cgroups; unmounting any cgroup roots [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPING [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) shut down [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Stopped thread '_TimeoutMonitor'. [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPED [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPING [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9001)) already shut down [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE No thread running for None. [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus STOPPED [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus EXITING [23/Jun/2022 14:17:05 +0000] 4681 MainThread _cplogging INFO [23/Jun/2022:14:17:05] ENGINE Bus EXITED [23/Jun/2022 14:17:06 +0000] 4681 MainThread agent INFO Agent exiting; caught signal 15 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO ================================================================================ [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO SCM Agent Version: 6.3.1 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent Protocol Version: 4 [23/Jun/2022 14:17:07 +0000] 5352 MainThread __init__ INFO Agent UUID file was last modified at 2022-06-22 15:28:23.208587 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Using Host ID: 788cfe56-58f9-4c12-a5a1-53500bd6bf9f [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Using directory: /run/cloudera-scm-agent [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Using supervisor binary path: /opt/cloudera/cm-agent/bin/../bin/supervisord [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent Logging Level: INFO [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent config: [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Security.use_tls = 0 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Security.max_cert_depth = 9 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.orphan_process_dir_staleness_threshold = 5184000 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.dns_resolution_collection_timeout_seconds = 30 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.orphan_process_dir_refresh_interval = 3600 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.local_filesystem_whitelist = ext2,ext3,ext4,xfs [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.stacks_log_bundle_max_bytes = 1073741824 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.metrics_url_timeout_seconds = 30.0 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.max_collection_wait_seconds = 10.0 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.task_metrics_timeout_seconds = 5.0 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.stacks_log_max_uncompressed_file_size_bytes = 5242880 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.scm_debug = INFO [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.server_host = 172.20.1.140 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.monitored_nodev_filesystem_types = nfs,nfs4,tmpfs [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.listening_hostname = cdh-manager [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.server_port = 7182 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.dns_resolution_collection_interval_seconds = 60 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO General.impala_profile_bundle_max_bytes = 1073741824 [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Neither verify_cert_file nor verify_cert_dir are configured. Not performing validation of server certificates in HTTPS communication. These options can be configured in this agent's config.ini file to enable certificate validation. [23/Jun/2022 14:17:07 +0000] 5352 MainThread agent INFO Agent starting as pid 5352 user root(0) group root(0). [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem blkio at /sys/fs/cgroup/blkio [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem devices at /sys/fs/cgroup/devices [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem cpuacct at /sys/fs/cgroup/cpu,cpuacct [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem cpu at /sys/fs/cgroup/cpu,cpuacct [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found existing subsystem memory at /sys/fs/cgroup/memory [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: cpu [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'writable_cgroup_dot_procs': True, 'default_cpu_shares': 1024, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True} [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: cpuacct [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_cpuacct': True} [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: memory [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_memory': True, 'default_memory_limit_in_bytes': 8796093022207, 'default_memory_soft_limit_in_bytes': 8796093022207} [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: devices [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_devices': True} [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Found cgroups subsystem: blkio [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Adding more caps: {'has_blkio': True, 'default_blkio_weight': 1000} [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_memory_limit_in_bytes [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_blkio_weight [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: writable_cgroup_dot_procs [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_cpu_rt_runtime_us [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_memory_soft_limit_in_bytes [23/Jun/2022 14:17:09 +0000] 5352 MainThread cgroups INFO Ignoring cgroup avro_cap: default_cpu_shares [23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Found cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes': 8796093022207, 'default_blkio_weight': 1000, 'writable_cgroup_dot_procs': True, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True, 'default_memory_soft_limit_in_bytes': 8796093022207, 'has_cpuacct': True, 'default_cpu_shares': 1024, 'has_devices': True, 'has_blkio': True} [23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] [23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Using timeout of 2.000000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Using join timeout of 0.100000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Using tolerance of 60.000000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] [23/Jun/2022 14:17:09 +0000] 5352 MainThread kt_renewer INFO Agent wide credential cache set to /var/run/cloudera-scm-agent/krb5cc_cm_agent_0 [23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Setting up supervisord event monitor. [23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Using metrics_url_timeout_seconds of 30.000000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread metrics INFO Importing tasktracker metric schema from file /opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/tasktracker/schema.json [23/Jun/2022 14:17:09 +0000] 5352 MainThread ntp_monitor INFO Using timeout of 2.000000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread dns_names INFO Using timeout of 30.000000 [23/Jun/2022 14:17:09 +0000] 5352 MainThread __init__ INFO Created DNS monitor. [23/Jun/2022 14:17:09 +0000] 5352 MainThread stacks_collection_manager INFO Using max_uncompressed_file_size_bytes: 5242880 [23/Jun/2022 14:17:09 +0000] 5352 MainThread daemon INFO Importing metric schema from file /opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/schema.json [23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CMF_PACKAGE_DIR': '/opt/cloudera/cm-agent/service', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUSTEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/opt/cloudera/cm/lib/postgresql-42.1.4.jre7.jar', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_PARQUET_HOME': '/usr/lib/parquet', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/opt/cloudera/cm', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_SERVER_ROOT': '/opt/cloudera/cm', 'CDH_KUDU_HOME': '/usr/lib/kudu', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_HUE_HOME': '/usr/lib/hue', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} [23/Jun/2022 14:17:09 +0000] 5352 MainThread agent INFO To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. [23/Jun/2022 14:17:11 +0000] 5352 MainThread supervisor INFO Trying to connect to supervisor (Attempt 1) [23/Jun/2022 14:17:11 +0000] 5352 MainThread supervisor INFO Supervisor version: 3.0, pid: 1129 [23/Jun/2022 14:17:11 +0000] 5352 MainThread supervisor INFO Successfully connected to supervisor [23/Jun/2022 14:17:11 +0000] 5352 MainThread agent INFO Supervisor version: 3.0, pid: 1129 [23/Jun/2022 14:17:11 +0000] 5352 MainThread agent INFO Connecting to previous supervisor: agent-1129-1655964031. [23/Jun/2022 14:17:13 +0000] 5352 MainThread supervisor INFO Triggering supervisord update. [23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Bus STARTING [23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Started monitor thread '_TimeoutMonitor'. [23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Serving on http://127.0.0.1:9001 [23/Jun/2022 14:17:13 +0000] 5352 MainThread _cplogging INFO [23/Jun/2022:14:17:13] ENGINE Bus STARTED [23/Jun/2022 14:17:15 +0000] 5352 MainThread daemon INFO New monitor: (<cmf.monitor.host.HostMonitor object at 0x7f3495dc4450>,) [23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Setting default socket timeout to 45 [23/Jun/2022 14:17:15 +0000] 5352 MonitorDaemon-Scheduler daemon INFO Monitor ready to report: ('HostMonitor',) [23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Previously active parcels: {} [23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Loading last saved hb response to complete initialization: /var/lib/cloudera-scm-agent/response.avro [23/Jun/2022 14:17:15 +0000] 5352 Monitor-HostMonitor network_interfaces INFO NIC iface eth0 doesn't support ETHTOOL (95) [23/Jun/2022 14:17:15 +0000] 5352 MainThread heartbeat_tracker INFO HB stats (seconds): num:1 LIFE_MIN:0.02 min:0.02 mean:0.02 max:0.02 LIFE_MAX:0.02 [23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO CM server guid: 78d0ba5f-6d53-4812-af67-6607fda2a71b [23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels [23/Jun/2022 14:17:15 +0000] 5352 MainThread parcel INFO Agent does create users/groups and apply file permissions [23/Jun/2022 14:17:15 +0000] 5352 MainThread downloader INFO Downloader path: /opt/cloudera/parcel-cache [23/Jun/2022 14:17:15 +0000] 5352 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache [23/Jun/2022 14:17:15 +0000] 5352 MainThread throttling_logger WARNING Failed parsing alternatives line: pax string index out of range link currently points to /usr/bin/spax [23/Jun/2022 14:17:15 +0000] 5352 MainThread agent INFO Flood daemon (re)start attempt [23/Jun/2022 14:17:17 +0000] 5352 MainThread agent INFO Reloading status server config, sending SIGHUP [23/Jun/2022 14:17:17 +0000] 5352 MainThread throttling_logger INFO Identified java component java8 with full version java version "1.8.0_301" Java(TM) SE Runtime Environment (build 1.8.0_301-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.301-b09, mixed mode) for requested version . [23/Jun/2022 14:17:45 +0000] 5352 DnsResolutionMonitor throttling_logger WARNING hostname cdh-manager differs from the canonical name iZuf60e0xat656rqmz890jZ
... View more
Labels:
- Labels:
-
Cloudera Manager