User can view entire hdfs dir and navigate further via WebHDFS. hadoop-policy (Access Control Lists) does not seem to be applicable to WebHDFS. how to incorporate ACLs when accessed via WebHDFS?

[#1]

User can view entire hdfs dir and navigate more via WebHDFS. hadoop-policy (Access Control Lists) does not seem to be applicable to WebHDFS. how to incorporate ACLs when accessed via WebHDFS?

[#2]

Are you referring to the hadoop-policy section in core-site and hdfs-site? These do not control security the way you'd expect. For proper ACLs on HDFS do either of these:

1. Secure (Kerberize) your cluster. Ambari automates this. Add Ranger and enable HDFS policies.
2. If accessing via REST API (WebHDFS) - restrict direct datanode access via a firewall and only allow access via Knox. Knox, in turn, will be able to map an incoming user into an actual role (still, full control with audit will require adding Ranger).

Andrew

[#3 · Accepted solution]

The ACLs specified in the hadoop-policy.xml file refer to Hadoop service-level authorization.

http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/ServiceLevelAuth.html

These ACLs are enforced on Hadoop RPC service calls. These ACLs are not applicable to access through WebHDFS. In order to fully control authorization to HDFS files, use HDFS permissions and ACLs.

http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html

Permissions and ACLs applied to directories and files are enforced for all means of access to the file system.

Other potential solutions are to use Knox or Ranger.

[#4]

@Saumil Mayani has this been resolved? Can you accept the best answer or provide your own solution?