How to enable deny-conditions and excludes in Ranger policies

[#1]

In HDP 2.5, the addition of RANGER-606 has introduced the ability to explicity deny access to a Hadoop resource via a Ranger Policy.

RANGER-876 makes these types of policies optional by default for all except tag-based policies.

To enable them, you must set enableDenyAndExceptionsInPolicies to true in the Service Definition for each of the Ranger Repository types as below, via the REST API:

{  
 "name": "hdfs",
 "description": "HDFS Repository",  
 "options": {
  "enableDenyAndExceptionsInPolicies": "true"  
 }
}

How To

If deny policies are not enabled, the Ranger “Create Policy” UI will look like this:

1. Get the current service definition of the desired repository via a curl command and output to a file:

curl -u admin:admin ranger-admin-host.hortonworks.com:6080/service/public/v2/api/servicedef/1 > hdfs.json

It should look something like this:

{"id":1,"guid":"0d047247-bafe-4cf8-8e9b-d5d377284b2d","isEnabled":true,"createTime":1476173228000,"updateTime":1476173228000,"version":1,"name":"hdfs","implClass":"org.apache.ranger.services.hdfs.RangerServiceHdfs","label":"HDFS Repository","description":"HDFS Repository","options":{},"configs":[{"itemId":1,"name":"username","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Username"},{"itemId":2,"name":"password","type":"password","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Password"},{"itemId":3,"name":"fs.default.name","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Namenode URL"},{"itemId":4,"name":"hadoop.security.authorization","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authorization Enabled"},{"itemId":5,"name":"hadoop.security.authentication","type":"enum","subType":"authnType","mandatory":true,"defaultValue":"simple","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authentication Type"},{"itemId":6,"name":"hadoop.security.auth_to_local","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":7,"name":"dfs.datanode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":8,"name":"dfs.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":9,"name":"dfs.secondary.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":10,"name":"hadoop.rpc.protection","type":"enum","subType":"rpcProtection","mandatory":false,"defaultValue":"authentication","validationRegEx":"","validationMessage":"","uiHint":"","label":"RPC Protection Type"},{"itemId":11,"name":"commonNameForCertificate","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Common Name for Certificate"}],"resources":[{"itemId":1,"name":"path","type":"path","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":true,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerPathResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"false"},"validationRegEx":"","validationMessage":"","uiHint":"","label":"Resource Path","description":"HDFS file or directory path"}],"accessTypes":[{"itemId":1,"name":"read","label":"Read","impliedGrants":[]},{"itemId":2,"name":"write","label":"Write","impliedGrants":[]},{"itemId":3,"name":"execute","label":"Execute","impliedGrants":[]}],"policyConditions":[],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authnType","elements":[{"itemId":1,"name":"simple","label":"Simple"},{"itemId":2,"name":"kerberos","label":"Kerberos"}],"defaultIndex":0},{"itemId":2,"name":"rpcProtection","elements":[{"itemId":1,"name":"authentication","label":"Authentication"},{"itemId":2,"name":"integrity","label":"Integrity"},{"itemId":3,"name":"privacy","label":"Privacy"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]}}

2. Update the file to add "options":{"enableDenyAndExceptionsInPolicies":"true"}

{"id":1,"guid":"0d047247-bafe-4cf8-8e9b-d5d377284b2d","isEnabled":true,"createdBy":"Admin","updatedBy":"Admin","createTime":1476173228000,"updateTime":1476287031622,"version":2,"name":"hdfs","implClass":"org.apache.ranger.services.hdfs.RangerServiceHdfs","label":"HDFS Repository","description":"HDFS Repository","options":{"enableDenyAndExceptionsInPolicies":"true"},"configs":[{"itemId":1,"name":"username","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Username"},{"itemId":2,"name":"password","type":"password","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Password"},{"itemId":3,"name":"fs.default.name","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Namenode URL"},{"itemId":4,"name":"hadoop.security.authorization","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authorization Enabled"},{"itemId":5,"name":"hadoop.security.authentication","type":"enum","subType":"authnType","mandatory":true,"defaultValue":"simple","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authentication Type"},{"itemId":6,"name":"hadoop.security.auth_to_local","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":7,"name":"dfs.datanode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":8,"name":"dfs.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":9,"name":"dfs.secondary.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":10,"name":"hadoop.rpc.protection","type":"enum","subType":"rpcProtection","mandatory":false,"defaultValue":"authentication","validationRegEx":"","validationMessage":"","uiHint":"","label":"RPC Protection Type"},{"itemId":11,"name":"commonNameForCertificate","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Common Name for Certificate"}],"resources":[{"itemId":1,"name":"path","type":"path","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":true,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerPathResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"false"},"validationRegEx":"","validationMessage":"","uiHint":"","label":"Resource Path","description":"HDFS file or directory path"}],"accessTypes":[{"itemId":1,"name":"read","label":"Read","impliedGrants":[]},{"itemId":2,"name":"write","label":"Write","impliedGrants":[]},{"itemId":3,"name":"execute","label":"Execute","impliedGrants":[]}],"policyConditions":[],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authnType","elements":[{"itemId":1,"name* Connection #0 to host ana-sme-security2.field.hortonworks.com left intact
":"simple","label":"Simple"},{"itemId":2,"name":"kerberos","label":"Kerberos"}],"defaultIndex":0},{"itemId":2,"name":"rpcProtection","elements":[{"itemId":1,"name":"authentication","label":"Authentication"},{"itemId":2,"name":"integrity","label":"Integrity"},{"itemId":3,"name":"privacy","label":"Privacy"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]}}

3. Put the updated file back into the Service Definition:

curl -iv -u admin:admin -X PUT -H "Accept: application/json" -H "Content-Type: application/json" -d @hdfs.json ranger-admin-host.hortonworks.com:6080/service/public/v2/api/servicedef/1

If successful, the Ranger “Create Policy” UI will look like this:

4. Repeat for any other desired repository.

References

Apache Ranger Wiki: Deny Conditions and Excludes in Ranger Policies

Apache Ranger Wiki: REST APIs for Service Definition, Service and Policy Management

[#2]

@Ana Gillan

For some reason this is not working for the hive service:

curl -u admin:admin sandbox.hortonworks.com:6080/service/public/v2/api/servicedef/3 > hive.json

curl -iv -u admin:admin -X PUT -H "Accept: application/json" -H "Content-Type: application/json" -d @hive.json sandbox.hortonworks.com:6080/service/public/v2/api/servicedef/3

< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Server: Apache-Coyote/1.1
Server: Apache-Coyote/1.1
< Set-Cookie: RANGERADMINSESSIONID=6E8B950D783DAA6524BF72850E0EF62E; Path=/; HttpOnly
Set-Cookie: RANGERADMINSESSIONID=6E8B950D783DAA6524BF72850E0EF62E; Path=/; HttpOnly
< X-Frame-Options: DENY
X-Frame-Options: DENY
< Content-Length: 0
Content-Length: 0
< Date: Mon, 17 Oct 2016 21:54:02 GMT
Date: Mon, 17 Oct 2016 21:54:02 GMT
< Connection: close
Connection: close

[#3]

Hi @Jasper sorry just spotted this comment. That's interesting - I used the same technique for all of them. Did you get it working since then? Since you will have downloaded it in the first curl I'm guessing the URL is right. Silly question, but is the hive plugin enabled?

[#4]

@Ana Gillan No it worked now. It was an error in the json file that causes the 404.

[#5]

@Ana Gillan

This approach works! I have done the HDFS and Hive service now. There is always something to wish for it seems 🙂

For Hive, it turns out that the deny/allow conditions extension only appears at the "Access" tab of the Hive policies, not the "Masking" and the "Row Filter" tabs. Can this be done as well? Or is it on the roadmap?

[#6]

@Ana Gillan

Thank you so much for this post its clear unlike others.

[#7]

This procedure has been replaced by https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/about_ranger_policies.h... . It is located on a page about implementing tag-based policies. ranger enableDenyAndExceptionsInPolicies=true deny and exception.