Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Namenode Alert -

Highlighted

Namenode Alert -

New Contributor

Description:

We found follow error message from ambari alert, It looks like could not connect namenode server (dwhdpm01p.xxx.com)

through following port (8019,50070).I have no idea, Please gives me some advice.

Check List :

1.netstat

[root@dwhdpm01p ~]# netstat -anp |grep 50070

tcp 0 0 192.168.1.160:50070 0.0.0.0:* LISTEN 10670/java

tcp 0 0 192.168.1.160:50070 192.168.1.161:54986 TIME_WAIT -

tcp6 0 0 172.16.7.40:8080 172.20.249.15:50070 ESTABLISHED 22260/java

[root@dwhdpm01p ~]# netstat -anp |grep 8019

tcp 0 0 192.168.1.160:8019 0.0.0.0:* LISTEN 10246/java

2.telnet

[root@dwhdpm01p ~]# telnet 192.168.1.160 50070

Trying 192.168.1.160... Connected to 192.168.1.160. Escape character is '^]'.

3.selinux close

[root@dwhdpm01p ~]# getenforce

Disabled

4.Iptables close

[root@dwhdpm01p ~]# systemctl status iptables

● iptables.service - IPv4 firewall with iptables

Loaded: loaded (/usr/lib/systemd/system/iptables.service; disabled; vendor preset: disabled)

Active: inactive (dead)

5.Webhdfs is work

curl -i "http://dwhdpm01p.xxx.com:50070/webhdfs/v1/user/admin/?op=LISTSTATUS"

Ambari Alert :

1.ZooKeeper Failover Controller Process

Connection failed: [Errno 111] Connection refused to dwhdpm01p.xxx.com:8019

2. NameNode Web UI

Connection failed to http://dwhdpm01p.xxx.com:50070 (timed out)
				

3. Ambari Server Alerts

There are 41 stale alerts from 1 host(s):
dwhdpm01p.xxx.com
  [Ambari Agent Distro/Conf Select Versions (8d 23h 37m),
  App Timeline Web UI (8d 23h 33m),
  DataNode Health Summary (8d 23h 33m),
  Grafana Web UI (8d 23h 32m),
  HDFS Capacity Utilization (8d 23h 33m),
  HDFS Pending Deletion Blocks (8d 23h 33m),
  HDFS Storage Capacity Usage (Daily) (9d 5h 47m),
  HDFS Storage Capacity Usage (Weekly) (9d 21h 47m),
  HDFS Upgrade Finalized State (8d 23h 33m),
  History Server CPU Utilization (8d 23h 37m),
  History Server Process (8d 23h 32m),
  History Server RPC Latency (8d 23h 37m),
  History Server Web UI (8d 23h 33m),
  Hive Metastore Process (8d 23h 34m),
  Host Disk Usage (8d 23h 33m),
  Metrics Collector - Auto-Restart Status (8d 23h 33m),
  Metrics Collector - HBase Master CPU Utilization (8d 23h 37m),
  Metrics Collector - HBase Master Process (8d 23h 33m),
  Metrics Collector Process (8d 23h 33m),
  Metrics Monitor Status (8d 23h 33m),
  NameNode Blocks Health (8d 23h 33m),
  NameNode Client RPC Processing Latency (Daily) (9d 5h 47m),
  NameNode Client RPC Processing Latency (Hourly) (8d 23h 37m),
  NameNode Client RPC Queue Latency (Daily) (9d 5h 47m),
  NameNode Client RPC Queue Latency (Hourly) (8d 23h 37m),
  NameNode Directory Status (8d 23h 33m),
  NameNode Heap Usage (Daily) (9d 5h 47m),
  NameNode Heap Usage (Weekly) (9d 21h 47m),
  NameNode Host CPU Utilization (8d 23h 37m),
  NameNode Last Checkpoint (8d 23h 33m),
  NameNode RPC Latency (8d 23h 33m),
  NameNode Service RPC Processing Latency (Daily) (9d 5h 47m),
  NameNode Service RPC Processing Latency (Hourly) (8d 23h 37m),
  NameNode Service RPC Queue Latency (Daily) (9d 5h 47m),
  NameNode Service RPC Queue Latency (Hourly) (8d 23h 37m),
  NameNode Web UI (8d 23h 33m),
  NodeManager Health Summary (8d 23h 33m),
  ResourceManager CPU Utilization (8d 23h 37m),
  ResourceManager RPC Latency (8d 23h 37m),
  ResourceManager Web UI (8d 23h 33m),
  ZooKeeper Failover Controller Process (8d 23h 33m)]
1 REPLY 1

Re: Namenode Alert -

New Contributor

Additional

Version :

ambari-server 2.5.2

HDP 2.4

Compenent Location:

Ambari-server install on dwhdpm01p.xxx.com.

Don't have an account?
Coming from Hortonworks? Activate your account here