Created on 10-12-2016 04:59 PM - edited 08-17-2019 08:49 AM
In HDP 2.5, the addition of RANGER-606 has introduced the ability to explicity deny access to a Hadoop resource via a Ranger Policy.
RANGER-876 makes these types of policies optional by default for all except tag-based policies.
To enable them, you must set enableDenyAndExceptionsInPolicies to true in the Service Definition for each of the Ranger Repository types as below, via the REST API:
{ "name": "hdfs", "description": "HDFS Repository", "options": { "enableDenyAndExceptionsInPolicies": "true" } }
How To
If deny policies are not enabled, the Ranger “Create Policy” UI will look like this:
curl -u admin:admin ranger-admin-host.hortonworks.com:6080/service/public/v2/api/servicedef/1 > hdfs.json
It should look something like this:
{"id":1,"guid":"0d047247-bafe-4cf8-8e9b-d5d377284b2d","isEnabled":true,"createTime":1476173228000,"updateTime":1476173228000,"version":1,"name":"hdfs","implClass":"org.apache.ranger.services.hdfs.RangerServiceHdfs","label":"HDFS Repository","description":"HDFS Repository","options":{},"configs":[{"itemId":1,"name":"username","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Username"},{"itemId":2,"name":"password","type":"password","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Password"},{"itemId":3,"name":"fs.default.name","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Namenode URL"},{"itemId":4,"name":"hadoop.security.authorization","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authorization Enabled"},{"itemId":5,"name":"hadoop.security.authentication","type":"enum","subType":"authnType","mandatory":true,"defaultValue":"simple","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authentication Type"},{"itemId":6,"name":"hadoop.security.auth_to_local","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":7,"name":"dfs.datanode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":8,"name":"dfs.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":9,"name":"dfs.secondary.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":10,"name":"hadoop.rpc.protection","type":"enum","subType":"rpcProtection","mandatory":false,"defaultValue":"authentication","validationRegEx":"","validationMessage":"","uiHint":"","label":"RPC Protection Type"},{"itemId":11,"name":"commonNameForCertificate","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Common Name for Certificate"}],"resources":[{"itemId":1,"name":"path","type":"path","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":true,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerPathResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"false"},"validationRegEx":"","validationMessage":"","uiHint":"","label":"Resource Path","description":"HDFS file or directory path"}],"accessTypes":[{"itemId":1,"name":"read","label":"Read","impliedGrants":[]},{"itemId":2,"name":"write","label":"Write","impliedGrants":[]},{"itemId":3,"name":"execute","label":"Execute","impliedGrants":[]}],"policyConditions":[],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authnType","elements":[{"itemId":1,"name":"simple","label":"Simple"},{"itemId":2,"name":"kerberos","label":"Kerberos"}],"defaultIndex":0},{"itemId":2,"name":"rpcProtection","elements":[{"itemId":1,"name":"authentication","label":"Authentication"},{"itemId":2,"name":"integrity","label":"Integrity"},{"itemId":3,"name":"privacy","label":"Privacy"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]}}
2. Update the file to add "options":{"enableDenyAndExceptionsInPolicies":"true"}
{"id":1,"guid":"0d047247-bafe-4cf8-8e9b-d5d377284b2d","isEnabled":true,"createdBy":"Admin","updatedBy":"Admin","createTime":1476173228000,"updateTime":1476287031622,"version":2,"name":"hdfs","implClass":"org.apache.ranger.services.hdfs.RangerServiceHdfs","label":"HDFS Repository","description":"HDFS Repository","options":{"enableDenyAndExceptionsInPolicies":"true"},"configs":[{"itemId":1,"name":"username","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Username"},{"itemId":2,"name":"password","type":"password","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Password"},{"itemId":3,"name":"fs.default.name","type":"string","subType":"","mandatory":true,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Namenode URL"},{"itemId":4,"name":"hadoop.security.authorization","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authorization Enabled"},{"itemId":5,"name":"hadoop.security.authentication","type":"enum","subType":"authnType","mandatory":true,"defaultValue":"simple","validationRegEx":"","validationMessage":"","uiHint":"","label":"Authentication Type"},{"itemId":6,"name":"hadoop.security.auth_to_local","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":7,"name":"dfs.datanode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":8,"name":"dfs.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":9,"name":"dfs.secondary.namenode.kerberos.principal","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":""},{"itemId":10,"name":"hadoop.rpc.protection","type":"enum","subType":"rpcProtection","mandatory":false,"defaultValue":"authentication","validationRegEx":"","validationMessage":"","uiHint":"","label":"RPC Protection Type"},{"itemId":11,"name":"commonNameForCertificate","type":"string","subType":"","mandatory":false,"validationRegEx":"","validationMessage":"","uiHint":"","label":"Common Name for Certificate"}],"resources":[{"itemId":1,"name":"path","type":"path","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":true,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerPathResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"false"},"validationRegEx":"","validationMessage":"","uiHint":"","label":"Resource Path","description":"HDFS file or directory path"}],"accessTypes":[{"itemId":1,"name":"read","label":"Read","impliedGrants":[]},{"itemId":2,"name":"write","label":"Write","impliedGrants":[]},{"itemId":3,"name":"execute","label":"Execute","impliedGrants":[]}],"policyConditions":[],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authnType","elements":[{"itemId":1,"name* Connection #0 to host ana-sme-security2.field.hortonworks.com left intact ":"simple","label":"Simple"},{"itemId":2,"name":"kerberos","label":"Kerberos"}],"defaultIndex":0},{"itemId":2,"name":"rpcProtection","elements":[{"itemId":1,"name":"authentication","label":"Authentication"},{"itemId":2,"name":"integrity","label":"Integrity"},{"itemId":3,"name":"privacy","label":"Privacy"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]}}
3. Put the updated file back into the Service Definition:
curl -iv -u admin:admin -X PUT -H "Accept: application/json" -H "Content-Type: application/json" -d @hdfs.json ranger-admin-host.hortonworks.com:6080/service/public/v2/api/servicedef/1
If successful, the Ranger “Create Policy” UI will look like this:
4. Repeat for any other desired repository.
References
Apache Ranger Wiki: Deny Conditions and Excludes in Ranger Policies
Apache Ranger Wiki: REST APIs for Service Definition, Service and Policy Management
Created on 10-17-2016 09:57 PM
For some reason this is not working for the hive service:
curl -u admin:admin sandbox.hortonworks.com:6080/service/public/v2/api/servicedef/3 > hive.json
curl -iv -u admin:admin -X PUT -H "Accept: application/json" -H "Content-Type: application/json" -d @hive.json sandbox.hortonworks.com:6080/service/public/v2/api/servicedef/3
< HTTP/1.1 404 Not Found HTTP/1.1 404 Not Found < Server: Apache-Coyote/1.1 Server: Apache-Coyote/1.1 < Set-Cookie: RANGERADMINSESSIONID=6E8B950D783DAA6524BF72850E0EF62E; Path=/; HttpOnly Set-Cookie: RANGERADMINSESSIONID=6E8B950D783DAA6524BF72850E0EF62E; Path=/; HttpOnly < X-Frame-Options: DENY X-Frame-Options: DENY < Content-Length: 0 Content-Length: 0 < Date: Mon, 17 Oct 2016 21:54:02 GMT Date: Mon, 17 Oct 2016 21:54:02 GMT < Connection: close Connection: close
Created on 10-20-2016 04:50 PM
Hi @Jasper sorry just spotted this comment. That's interesting - I used the same technique for all of them. Did you get it working since then? Since you will have downloaded it in the first curl I'm guessing the URL is right. Silly question, but is the hive plugin enabled?
Created on 10-20-2016 09:24 PM
@Ana Gillan No it worked now. It was an error in the json file that causes the 404.
Created on 10-20-2016 09:28 PM
This approach works! I have done the HDFS and Hive service now. There is always something to wish for it seems 🙂
For Hive, it turns out that the deny/allow conditions extension only appears at the "Access" tab of the Hive policies, not the "Masking" and the "Row Filter" tabs. Can this be done as well? Or is it on the roadmap?
Created on 09-12-2017 10:08 AM
Thank you so much for this post its clear unlike others.
Created on 09-26-2017 08:54 PM
This procedure has been replaced by https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/about_ranger_policies.h... . It is located on a page about implementing tag-based policies. ranger enableDenyAndExceptionsInPolicies=true deny and exception.