Reply
Contributor
Posts: 45
Registered: ‎01-24-2016

This host has been out of contact with the Cloudera Manager Server for too long

Hi guys 

 

I face this problem every day now on at least two of the hosts 

 

 

"This host has been out of contact with the Cloudera Manager Server for too long. This host is in contact with the Host Monitor. The host's Cloudera Manager Agent version matches the Host Monitor version (5.8.1)."

 

Inspector Results Validations
 Inspector failed on the following hosts...
  • ip-10-231-19-[23, 196]: Can only run host inspector when host is healthy.
Inspector ran on 6 hosts.
 Individual hosts resolved their own hostnames correctly.
 No errors were found while looking for conflicting init scripts.
 No errors were found while checking /etc/hosts.
 All hosts resolved localhost to 127.0.0.1.
 All hosts checked resolved each other's hostnames correctly and in a timely manner.
 Host clocks are approximately in sync (within ten minutes).
 Host time zones are consistent across the cluster.
 No users or groups are missing.
 No conflicts detected between packages and parcels.
 No kernel versions that are known to be bad are running.
 No problems were found with /proc/sys/vm/swappiness on any of the hosts.
 No performance concerns with Transparent Huge Pages settings.
 CDH 5 Hue Python version dependency is satisfied.
 0 hosts are running CDH 4 and 8 hosts are running CDH 5.
 There are mismatched versions across the system, which will cause failures. See below for details on which hosts are running what versions of components.
 Java versions are inconsistent across the managed hosts. Check the component version below to identify hosts with inconsistent Java versions.
 All checked Cloudera Management Daemons versions are consistent with the server.
 All checked Cloudera Management Agents versions are consistent with the server.
Version Summary HostsComponent Version Release CDH Version
ml-nonprod — Group 1 (CDH 5)
ip-10-231-18-[68, 203]; ip-10-231-19-[23, 86-87, 196]
Bigtop-Tomcat (CDH 5 only)0.7.0+cdh5.8.0+01.cdh5.8.0.p0.73CDH 5
Crunch (CDH 5 only)0.11.0+cdh5.8.0+911.cdh5.8.0.p0.77CDH 5
Flume NG1.6.0+cdh5.8.0+501.cdh5.8.0.p0.75CDH 5
MapReduce 12.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
Hadoop2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HDFS2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HttpFS2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
hadoop-kms2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
MapReduce 22.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
YARN2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HBase1.2.0+cdh5.8.0+1601.cdh5.8.0.p0.80CDH 5
Lily HBase Indexer1.5+cdh5.8.0+641.cdh5.8.0.p0.75CDH 5
Hive1.1.0+cdh5.8.0+6101.cdh5.8.0.p0.77CDH 5
HCatalog1.1.0+cdh5.8.0+6101.cdh5.8.0.p0.77CDH 5
Hue3.9.0+cdh5.8.0+25121.cdh5.8.0.p0.88CDH 5
Impala2.6.0+cdh5.8.0+01.cdh5.8.0.p0.111CDH 5
Kite (CDH 5 only)1.0.0+cdh5.8.0+1361.cdh5.8.0.p0.73CDH 5
Llama (CDH 5 only)1.0.0+cdh5.8.0+01.cdh5.8.0.p0.73CDH 5
Mahout0.9+cdh5.8.0+271.cdh5.8.0.p0.71CDH 5
Oozie4.1.0+cdh5.8.0+2911.cdh5.8.0.p0.83CDH 5
Parquet1.5.0+cdh5.8.0+1741.cdh5.8.0.p0.71CDH 5
Pig0.12.0+cdh5.8.0+831.cdh5.8.0.p0.71CDH 5
sentry1.5.1+cdh5.8.0+2441.cdh5.8.0.p0.83CDH 5
Solr4.10.3+cdh5.8.0+4231.cdh5.8.0.p0.79CDH 5
spark1.6.0+cdh5.8.0+2051.cdh5.8.0.p0.74CDH 5
Sqoop21.99.5+cdh5.8.0+381.cdh5.8.0.p0.72CDH 5
Sqoop1.4.6+cdh5.8.0+651.cdh5.8.0.p0.69CDH 5
Whirr0.9.0+cdh5.8.0+171.cdh5.8.0.p0.68CDH 5
Zookeeper3.4.5+cdh5.8.0+941.cdh5.8.0.p0.76CDH 5
Cloudera Manager Management Daemons5.8.11.cm581.p0.7Not applicable
Supervisord3.0-cm5.8.1UnavailableNot applicable
Java 7JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)UnavailableNot applicable
Java 6JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)UnavailableNot applicable
Cloudera Manager Agent5.8.11.cm581.p0.7Not applicable
HostsComponent Version Release CDH Version
ml-nonprod — Group 2 (CDH 5)
ip-10-231-19-197
Bigtop-Tomcat (CDH 5 only)0.7.0+cdh5.8.0+01.cdh5.8.0.p0.73CDH 5
Crunch (CDH 5 only)0.11.0+cdh5.8.0+911.cdh5.8.0.p0.77CDH 5
Flume NG1.6.0+cdh5.8.0+501.cdh5.8.0.p0.75CDH 5
MapReduce 12.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
Hadoop2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HDFS2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HttpFS2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
hadoop-kms2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
MapReduce 22.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
YARN2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HBase1.2.0+cdh5.8.0+1601.cdh5.8.0.p0.80CDH 5
Lily HBase Indexer1.5+cdh5.8.0+641.cdh5.8.0.p0.75CDH 5
Hive1.1.0+cdh5.8.0+6101.cdh5.8.0.p0.77CDH 5
HCatalog1.1.0+cdh5.8.0+6101.cdh5.8.0.p0.77CDH 5
Hue3.9.0+cdh5.8.0+25121.cdh5.8.0.p0.88CDH 5
Impala2.6.0+cdh5.8.0+01.cdh5.8.0.p0.111CDH 5
Kite (CDH 5 only)1.0.0+cdh5.8.0+1361.cdh5.8.0.p0.73CDH 5
Llama (CDH 5 only)1.0.0+cdh5.8.0+01.cdh5.8.0.p0.73CDH 5
Mahout0.9+cdh5.8.0+271.cdh5.8.0.p0.71CDH 5
Oozie4.1.0+cdh5.8.0+2911.cdh5.8.0.p0.83CDH 5
Parquet1.5.0+cdh5.8.0+1741.cdh5.8.0.p0.71CDH 5
Pig0.12.0+cdh5.8.0+831.cdh5.8.0.p0.71CDH 5
sentry1.5.1+cdh5.8.0+2441.cdh5.8.0.p0.83CDH 5
Solr4.10.3+cdh5.8.0+4231.cdh5.8.0.p0.79CDH 5
spark1.6.0+cdh5.8.0+2051.cdh5.8.0.p0.74CDH 5
Sqoop21.99.5+cdh5.8.0+381.cdh5.8.0.p0.72CDH 5
Sqoop1.4.6+cdh5.8.0+651.cdh5.8.0.p0.69CDH 5
Whirr0.9.0+cdh5.8.0+171.cdh5.8.0.p0.68CDH 5
Zookeeper3.4.5+cdh5.8.0+941.cdh5.8.0.p0.76CDH 5
Cloudera Manager Management Daemons5.8.11.cm581.p0.7Not applicable
Supervisord3.0-cm5.8.1UnavailableNot applicable
Java 7JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)UnavailableNot applicable
Java 6JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)UnavailableNot applicable
Java 8JAVA_HOME=/usr/lib/jvm/java-8-oracle java version "1.8.0_101" Java(TM) SE Runtime Environment (build 1.8.0_101-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)UnavailableNot applicable
Cloudera Manager Agent5.8.11.cm581.p0.7Not applicable
HostsComponent Version Release CDH Version
ml-nonprod — Group 3 (CDH 5)
ip-10-231-18-181
Bigtop-Tomcat (CDH 5 only)0.7.0+cdh5.8.0+01.cdh5.8.0.p0.73CDH 5
Crunch (CDH 5 only)0.11.0+cdh5.8.0+911.cdh5.8.0.p0.77CDH 5
Flume NG1.6.0+cdh5.8.0+501.cdh5.8.0.p0.75CDH 5
MapReduce 12.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
Hadoop2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HDFS2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HttpFS2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
hadoop-kms2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
MapReduce 22.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
YARN2.6.0+cdh5.8.0+16011.cdh5.8.0.p0.93CDH 5
HBase1.2.0+cdh5.8.0+1601.cdh5.8.0.p0.80CDH 5
Lily HBase Indexer1.5+cdh5.8.0+641.cdh5.8.0.p0.75CDH 5
Hive1.1.0+cdh5.8.0+6101.cdh5.8.0.p0.77CDH 5
HCatalog1.1.0+cdh5.8.0+6101.cdh5.8.0.p0.77CDH 5
Hue3.9.0+cdh5.8.0+25121.cdh5.8.0.p0.88CDH 5
Impala2.6.0+cdh5.8.0+01.cdh5.8.0.p0.111CDH 5
Kite (CDH 5 only)1.0.0+cdh5.8.0+1361.cdh5.8.0.p0.73CDH 5
Llama (CDH 5 only)1.0.0+cdh5.8.0+01.cdh5.8.0.p0.73CDH 5
Mahout0.9+cdh5.8.0+271.cdh5.8.0.p0.71CDH 5
Oozie4.1.0+cdh5.8.0+2911.cdh5.8.0.p0.83CDH 5
Parquet1.5.0+cdh5.8.0+1741.cdh5.8.0.p0.71CDH 5
Pig0.12.0+cdh5.8.0+831.cdh5.8.0.p0.71CDH 5
sentry1.5.1+cdh5.8.0+2441.cdh5.8.0.p0.83CDH 5
Solr4.10.3+cdh5.8.0+4231.cdh5.8.0.p0.79CDH 5
spark1.6.0+cdh5.8.0+2051.cdh5.8.0.p0.74CDH 5
Sqoop21.99.5+cdh5.8.0+381.cdh5.8.0.p0.72CDH 5
Sqoop1.4.6+cdh5.8.0+651.cdh5.8.0.p0.69CDH 5
Whirr0.9.0+cdh5.8.0+171.cdh5.8.0.p0.68CDH 5
Zookeeper3.4.5+cdh5.8.0+941.cdh5.8.0.p0.76CDH 5
Cloudera Manager Management Daemons5.8.11.cm581.p0.7Not applicable
Supervisord3.0-cm5.8.1UnavailableNot applicable
Java 7JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)UnavailableNot applicable
Java 6JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)UnavailableNot applicable
Java 8JAVA_HOME=/usr/lib/jvm/java-8-oracle java version "1.8.0_111" Java(TM) SE Runtime Environment (build 1.8.0_111-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)UnavailableNot applicable
Cloudera Manager Agent5.8.11.cm581.p0.7Not applicable
New Contributor
Posts: 2
Registered: ‎08-08-2017

Re: This host has been out of contact with the Cloudera Manager Server for too long

we have faced same issue in our cluster, after investigating the cloudera agent logs we got to know their was a process got hung.hence we have restrted the cloudera agent service and the issue got resolved.

Champion
Posts: 768
Registered: ‎05-16-2016

Re: This host has been out of contact with the Cloudera Manager Server for too long

Check if your ntp is synhronised 

Announcements