In Hadoop 2.7.2(CentOS 7) Cluster ,Datanode starts but doesn't connect to namenode

xunlei1221 — Thu, 23 Jun 2016 14:37:09 GMT

I installed a three node hadoop cluster. The master and slave node starts separately but the datanode isn't shown in namenode webUI. The log file for datanode shows the following error :

2016-06-18 21:23:53,980 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: namenode/192.168.1.100:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-06-18 21:23:55,029 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: namenode/192.168.1.100:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-06-18 21:23:56,030 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: namenode/192.168.1.100:9000. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-06-18 21:23:57,031 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: namenode/192.168.1.100:9000. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-06-18 21:23:58,032 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: namenode/192.168.1.100:9000. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

namenode machine's infomation:

cat /etc/hosts

#127.0.0.1   localhost localhost.localdomain localhost4            localhost4.localdomain4
#::1         localhost localhost.localdomain localhost6        localhost6.localdomain6
192.168.1.100 namenode
192.168.1.101 datanode1
192.168.1.102 datanode2

cat /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
IPV6INIT=yes
BOOTPROTO=dhcp
UUID=61fe61d3-fcda-4fed-ba81-bfa767e0270a
ONBOOT=yes
TYPE=Ethernet
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME="System eth0"
BOOTPROTO="static" 
ONBOOT="yes" 
IPADDR=192.168.1.100 
GATEWAY=192.168.1.1 
NETMASK=255.255.255.0 
DNS1=192.168.1.1

cat /etc/hostname

namenode

cat core-site.xml

<configuration>
<property>
    <name>hadoop.tmp.dir</name>
    <value>/home/hadoop/tmp</value>
    <description>Abase for other temporary directories.</description>
</property>
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://namenode:9000</value>
</property>
<property>
    <name>io.file.buffer.size</name>
    <value>4096</value>
</property>

cat hdfs-site.xml

<configuration>
<property>
    <name>dfs.nameservices</name>
    <value>hadoop-cluster1</value>
</property>
<property>
    <name>dfs.namenode.secondary.http-address</name>
    <value>namenode:50090</value>
</property>
<property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///home/hadoop/dfs/name</value>
</property>
<property>
    <name>dfs.datanode.data.dir</name>
    <value>file:///home/hadoop/dfs/data</value>
</property>
<property>
    <name>dfs.replication</name>
    <value>2</value>
</property>
<property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
</property>

cat mapred-site.xml

<configuration>
<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
</property>
<property>
    <name>mapreduce.jobtracker.http.address</name>
    <value>namenode:50030</value>
</property>
<property>
    <name>mapreduce.jobhistory.address</name>
    <value>namenode:10020</value>
</property>
<property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>namenode:19888</value>
</property>

cat yarn-site.xml

<configuration>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.resourcemanager.address</name>
    <value>namenode:8032</value>
</property>
<property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>namenode:8030</value>
</property>
<property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>namenode:8031</value>
</property>
<property>
    <name>yarn.resourcemanager.admin.address</name>
    <value>namenode:8033</value>
</property>
<property>
    <name>yarn.resourcemanager.webapp.address</name>
    <value>namenode:8088</value>
</property>

cat slaves

datanode1
datanode2

Re: In Hadoop 2.7.2(CentOS 7) Cluster ,Datanode starts but doesn't connect to namenode

jyadav — Thu, 23 Jun 2016 15:29:49 GMT

First please check if namenode is up and running. If yes, then from datanode host run below command and see if it able to connect to the NN port.

From DN:

telnet 192.168.1.100 9000

If telnet is not responding then you might have firewall rules blocking the connection. Try disabling firewall and see if that make any difference.

Re: In Hadoop 2.7.2(CentOS 7) Cluster ,Datanode starts but doesn't connect to namenode

xunlei1221 — Thu, 23 Jun 2016 20:18:58 GMT

you are right,the firewall is running

question In Hadoop 2.7.2(CentOS 7) Cluster ,Datanode starts but doesn't connect to namenode in Archives of Support Questions (Read Only)

In Hadoop 2.7.2(CentOS 7) Cluster ,Datanode starts but doesn't connect to namenode

Re: In Hadoop 2.7.2(CentOS 7) Cluster ,Datanode starts but doesn't connect to namenode

Re: In Hadoop 2.7.2(CentOS 7) Cluster ,Datanode starts but doesn't connect to namenode