Is Webhdfs supported in HDCloud Protected Gateway?

[#1]

Looking at the cluster console I see for network config:

PROTECTED GATEWAY ACCESSAmbari and Zeppelin Web UIs Hive JDBC Cluster Components (NameNode, ResourceManager, Spark History Server, Job History Server)
However, for https://hortonworks.github.io/hdp-aws/security-network/index.html it says that hdfs is supported as a gateway service.
When I try to browse hdfs via https://ec2-54-87-212-190.compute-1.amazonaws.com/cluster1/services/hdfs/explorer.html#/ I just get an error.

[#2]

Did you try port 443, per the page you referenced?

[#3]

HTTPS defaults to port 443

[#4]

@Tim Shephard

Are you using your custom VPC and subnet? If so, have you checked this checklist:

https://hortonworks.github.io/hdp-aws/security-vpc/index.html#configuring-your-own-vpc

Hope this helps!

[#5]

So are you saying that the knox gateway supports webhdfs? It's not listed on the UI in the cloud cluster console:

PROTECTED GATEWAY ACCESSAmbari and Zeppelin Web UIs Hive JDBC Cluster Components (NameNode, ResourceManager, Spark History Server, Job History Server)

[#6]

@Tim Shephard

No, I have not written anything related the webhdfs in my answer, but actually both hdfs ui and webhdfs is supported and should work if checked in UI.

Hope this helps!

[#7]

@pdarvasi that's a very cool reference. Great to see the source code. Unfortunately, I didn't see any option in the UI for

HDFSUI or for WEBHDFS, however I did see options for Ambari, Zeppelin, Hive, JDBC, NameNode, ResourceManager, Spark History Service, and Job History server. That's why I'm concluding that HDFS/WEBHDFS is not supported

[#8 · Accepted solution]

@Tim Shephard

You were right about WebHDFS not exposed, but HDFS UI is exposed via the NameNode service so it should be accessible if you enable "Protected Gateway Access to Cluster Components".

If you would like to enable WebHDFS, there is a workaround:

• SSH to master node of the cluster
• Edit /srv/pillar/gateway/init.sls
• Add "WEBHDFS" to gateway:exposed like below:

gateway:
  address: 172.21.250.198
  exposed: [WEBHDFS]
  location:
....

• After save, run salt '*' state.highstate (this will regenerate Knox topology)

Hope this helps!

[#9]

Wow, awesome. Thx!