question Re: NameNode running but 50070 not listening in Archives of Support Questions (Read Only)

NameNode running but 50070 not listening

Tomas79 — Thu, 30 Aug 2018 14:27:21 GMT

Hi,

having a non-ha setup of kerberized HDFS where the Namenode is running but

 netstat -an | grep 50070

does not return anything (running as root on namenode host)

The WebHDFS is enabled in CM:

dfs.webhdfs.enabled=True

Any hints why this port is not open and why the NN is not listening? Thanks

Re: NameNode running but 50070 not listening

EricL — Thu, 06 Sep 2018 06:52:05 GMT

50070 is HDFS's web UI port, it can be configured differently on each cluster.

You can check your hdfs-site.xml file and what's the value for dfs.namenode.http-address?

Re: NameNode running but 50070 not listening

Tomas79 — Thu, 06 Sep 2018 07:12:43 GMT

Thanks for the clarification, this is the hdfs-site.xml.

I am little bit confused, so the WebHDFS REST API is listening on the same port as the NameNode's UI?

Thanks

  <property>
    <name>dfs.ha.namenodes.hanameservice</name>
    <value>namenode18,namenode21</value>
  </property>
  <property>
    <name>dfs.namenode.rpc-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:8020</value>
  </property>
  <property>
    <name>dfs.namenode.servicerpc-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:8022</value>
  </property>
  <property>
    <name>dfs.namenode.http-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:50070</value>
  </property>
  <property>
    <name>dfs.namenode.https-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:50470</value>
  </property>
  <property>
    <name>dfs.namenode.rpc-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:8020</value>
  </property>
  <property>
    <name>dfs.namenode.servicerpc-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:8022</value>
  </property>
  <property>
    <name>dfs.namenode.http-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:50070</value>
  </property>
  <property>
    <name>dfs.namenode.https-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:50470</value>

Re: NameNode running but 50070 not listening

Harsh J — Thu, 06 Sep 2018 07:56:54 GMT

> I am little bit confused, so the WebHDFS REST API is listening on the same port as the NameNode's UI?

Yes this is correct. The HTTP(S) serving port of the NameNode does multiple things: Serves the UI for browsers on / and a few other paths, serves a REST API on /webhdfs/*, etc.

WebHDFS on HDFS service is used by contacting the currently configured web port of the NameNode and DataNode(s) (the latter by following redirects, not directly).

In your case, the cluster is set to use HTTPS (TLS security) so you need to use the 50470 port, swebhdfs:// (notice the s-prefix for security) in place of webhdfs:// and https:// in place of http:// when following any WebHDFS tutorial.