Support Questions

Find answers, ask questions, and share your expertise

NameNode running but 50070 not listening

avatar

Hi,

 having a non-ha setup of kerberized HDFS where the Namenode is running but 

 netstat -an | grep 50070

does not return anything (running as root on namenode host)

 The WebHDFS is enabled in CM:

dfs.webhdfs.enabled=True

 

Any hints why this port is not open and why the NN is not listening? Thanks

1 ACCEPTED SOLUTION

avatar
Mentor
> I am little bit confused, so the WebHDFS REST API is listening on the same port as the NameNode's UI?

Yes this is correct. The HTTP(S) serving port of the NameNode does multiple things: Serves the UI for browsers on / and a few other paths, serves a REST API on /webhdfs/*, etc.

WebHDFS on HDFS service is used by contacting the currently configured web port of the NameNode and DataNode(s) (the latter by following redirects, not directly).

In your case, the cluster is set to use HTTPS (TLS security) so you need to use the 50470 port, swebhdfs:// (notice the s-prefix for security) in place of webhdfs:// and https:// in place of http:// when following any WebHDFS tutorial.

View solution in original post

3 REPLIES 3

avatar
Super Guru
50070 is HDFS's web UI port, it can be configured differently on each cluster.

You can check your hdfs-site.xml file and what's the value for dfs.namenode.http-address?

avatar

Thanks for the clarification, this is the hdfs-site.xml.

I am little bit confused, so the WebHDFS REST API is listening on the same port as the NameNode's UI? 

Thanks

 

  <property>
    <name>dfs.ha.namenodes.hanameservice</name>
    <value>namenode18,namenode21</value>
  </property>
  <property>
    <name>dfs.namenode.rpc-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:8020</value>
  </property>
  <property>
    <name>dfs.namenode.servicerpc-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:8022</value>
  </property>
  <property>
    <name>dfs.namenode.http-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:50070</value>
  </property>
  <property>
    <name>dfs.namenode.https-address.hanameservice.namenode18</name>
    <value>ip-10-197-31-86.eu-west-1.compute.internal:50470</value>
  </property>
  <property>
    <name>dfs.namenode.rpc-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:8020</value>
  </property>
  <property>
    <name>dfs.namenode.servicerpc-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:8022</value>
  </property>
  <property>
    <name>dfs.namenode.http-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:50070</value>
  </property>
  <property>
    <name>dfs.namenode.https-address.hanameservice.namenode21</name>
    <value>ip-10-197-7-125.eu-west-1.compute.internal:50470</value>

avatar
Mentor
> I am little bit confused, so the WebHDFS REST API is listening on the same port as the NameNode's UI?

Yes this is correct. The HTTP(S) serving port of the NameNode does multiple things: Serves the UI for browsers on / and a few other paths, serves a REST API on /webhdfs/*, etc.

WebHDFS on HDFS service is used by contacting the currently configured web port of the NameNode and DataNode(s) (the latter by following redirects, not directly).

In your case, the cluster is set to use HTTPS (TLS security) so you need to use the 50470 port, swebhdfs:// (notice the s-prefix for security) in place of webhdfs:// and https:// in place of http:// when following any WebHDFS tutorial.