Skip to main content
iit2013121
iit2013121
Gold ⭐️
February 10, 2016
Solved

Resolving HDFS Connectivity: Changing NameNode Bind Address from Localhost to Hostname

I installed a 2 node hadoop cluster. The master and slave node starts separately but the datanode isn't shown in namenode webUI. The log file for datanode shows the following error :

2016-02-09 23:30:53,920 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-02-09 23:30:53,920 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2016-02-09 23:30:54,976 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop-master/172.17.25.5:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-02-09 23:30:55,977 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop-master/172.17.25.5:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2016-02-09 23:21:15,062 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/172.17.25.5:9000

The hosts file in slave is

172.17.25.5 hadoop-master
127.0.0.1 silp-ProLiant-DL360-Gen9
172.17.25.18 hadoop-slave-1
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

the core-site.xml file is :

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoop-master:9000</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>

Kindly help with this issue.

    nsabharwal
    nsabharwalAnswer
    Gold ⭐️
    February 10, 2016
    @Kumar Sanyam

    Is there connectivity between the servers?

    WARN org.apache.hadoop.hdfs.server.datanode.DataNode:Problem connecting to server: hadoop-master/172.17.25.5:9000

    Please make sure that ssh works and iptable is off

    View in thread

    12 replies

    nsabharwal
    nsabharwalAnswer
    Gold ⭐️
    February 10, 2016
    @Kumar Sanyam

    Is there connectivity between the servers?

    WARN org.apache.hadoop.hdfs.server.datanode.DataNode:Problem connecting to server: hadoop-master/172.17.25.5:9000

    Please make sure that ssh works and iptable is off

    Translate
      aervits
      aervits
      Gold ⭐️
      February 10, 2016
      @Kumar Sanyam

      turn off ipv6 in /etc/hosts.

      Translate
        Shelton
        Shelton
        Gold ⭐️⭐️⭐️
        February 10, 2016

        1. To add on to Artem's idea.

        Disable IPv6 Temporarily 'echo 1 > /proc/sys/net/ipv6/conf/<interface-name>/disable_ipv6'

        eg 'echo 1 > /proc/sys/net/ipv6/conf/eth0/disable_ipv6'

        Disable IPv6 Permanently across reboots

        Edit the /etc/sysctl.conf and the following lines.

        # to disable IPv6 on all interfaces system wide net.ipv6.conf.all.disable_ipv6 = 1

        # to disable IPv6 on a specific interface (e.g., eth0, lo)

        net.ipv6.conf.lo.disable_ipv6 = 1

        net.ipv6.conf.eth0.disable_ipv6 = 1

        2. Check the NTP server configuration or just restart NTP on both servers

        Translate
          ashneesharma88
          ashneesharma88
          Gold ⭐️
          February 17, 2016

          turn off ipv6 in /etc/sysconfig/network or /etc/sysconfif/network-manager/ifcfg-eth0

          parameter

          NETWORKING_IPV6=no

          IPV6INIT=no

          Please check selniux is disable or not if not then disable it and reboot.

          vi /etc/sysconfig/selinux

          selinux=disabled

          Also check ntp and iptable and firewall.

          Translate
            iit2013121
            iit2013121Author
            Gold ⭐️
            February 18, 2016

            thanks everyone..solved the problem. Actually the namenode was listening at localhost:9000 and datanode tried to connect at hadoop-master:9000, hence the error in connectivity. Changed the listening IP:port of namenode to hadoop-master:9000.

            Translate
              pradeepkumart
              pradeepkumart
              Silver ⭐️
              July 18, 2016

              I am facing the same issue. Could you please guide me how did you solve the problem. I am not able to get a clue from the answer.

              Translate
              dkant007
              dkant007
              Silver ⭐️
              December 14, 2018

              Thanks all for contributing to this post. I had the same issue on Centos 7 on a 2 node hadoop cluster. In my case firewall was the cause. After allowing my namenode port (in my case 8020) through firewall on namenode machine allowed datanodes to connect to namenode.

              $ sudo firewall-cmd --zone=public --add-port=8020/tcp --permanent
              $ sudo firewall-cmd --reload

              ,

              Thanks all for contributing to this post. I had the same issue on Centos 7 on a 2 node hadoop cluster. In my case firewall was the cause. After allowing my namenode port (in my case 8020) through firewall on namenode machine allowed datanodes to connect to namenode.

              $ sudo firewall-cmd --zone=public --add-port=8020/tcp --permanent
              $ sudo firewall-cmd --reload

              Translate
              ha_ahmadkhani
              ha_ahmadkhani
              Silver ⭐️⭐️
              May 4, 2019

              add port to firewall and enjoy


              sudo firewall-cmd --zone=public --add-port=8020/tcp --permanent
              sudo firewall-cmd --reload

              Translate
              Terms & ConditionsAccessibility statement