- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
NameNode running but 50070 not listening
- Labels:
-
Cloudera Manager
-
HDFS
-
Kerberos
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
having a non-ha setup of kerberized HDFS where the Namenode is running but
netstat -an | grep 50070
does not return anything (running as root on namenode host)
The WebHDFS is enabled in CM:
dfs.webhdfs.enabled=True
Any hints why this port is not open and why the NN is not listening? Thanks
Created ‎09-06-2018 12:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes this is correct. The HTTP(S) serving port of the NameNode does multiple things: Serves the UI for browsers on / and a few other paths, serves a REST API on /webhdfs/*, etc.
WebHDFS on HDFS service is used by contacting the currently configured web port of the NameNode and DataNode(s) (the latter by following redirects, not directly).
In your case, the cluster is set to use HTTPS (TLS security) so you need to use the 50470 port, swebhdfs:// (notice the s-prefix for security) in place of webhdfs:// and https:// in place of http:// when following any WebHDFS tutorial.
Created ‎09-05-2018 11:52 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can check your hdfs-site.xml file and what's the value for dfs.namenode.http-address?
Created ‎09-06-2018 12:12 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the clarification, this is the hdfs-site.xml.
I am little bit confused, so the WebHDFS REST API is listening on the same port as the NameNode's UI?
Thanks
<property> <name>dfs.ha.namenodes.hanameservice</name> <value>namenode18,namenode21</value> </property> <property> <name>dfs.namenode.rpc-address.hanameservice.namenode18</name> <value>ip-10-197-31-86.eu-west-1.compute.internal:8020</value> </property> <property> <name>dfs.namenode.servicerpc-address.hanameservice.namenode18</name> <value>ip-10-197-31-86.eu-west-1.compute.internal:8022</value> </property> <property> <name>dfs.namenode.http-address.hanameservice.namenode18</name> <value>ip-10-197-31-86.eu-west-1.compute.internal:50070</value> </property> <property> <name>dfs.namenode.https-address.hanameservice.namenode18</name> <value>ip-10-197-31-86.eu-west-1.compute.internal:50470</value> </property> <property> <name>dfs.namenode.rpc-address.hanameservice.namenode21</name> <value>ip-10-197-7-125.eu-west-1.compute.internal:8020</value> </property> <property> <name>dfs.namenode.servicerpc-address.hanameservice.namenode21</name> <value>ip-10-197-7-125.eu-west-1.compute.internal:8022</value> </property> <property> <name>dfs.namenode.http-address.hanameservice.namenode21</name> <value>ip-10-197-7-125.eu-west-1.compute.internal:50070</value> </property> <property> <name>dfs.namenode.https-address.hanameservice.namenode21</name> <value>ip-10-197-7-125.eu-west-1.compute.internal:50470</value>
Created ‎09-06-2018 12:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes this is correct. The HTTP(S) serving port of the NameNode does multiple things: Serves the UI for browsers on / and a few other paths, serves a REST API on /webhdfs/*, etc.
WebHDFS on HDFS service is used by contacting the currently configured web port of the NameNode and DataNode(s) (the latter by following redirects, not directly).
In your case, the cluster is set to use HTTPS (TLS security) so you need to use the 50470 port, swebhdfs:// (notice the s-prefix for security) in place of webhdfs:// and https:// in place of http:// when following any WebHDFS tutorial.
