question Re: Enabling and Disabling a Ranger Policy using command in Support Questions

Enabling and Disabling a Ranger Policy using command

rajdip_chaudhur — Thu, 06 Apr 2017 22:35:57 GMT

Hello,

We use Ranger for column filtering and data masking. Our use case is we will be ingesting data into Hive from source systems using Talend / Informatica but while executing the jobs we are getting error. What we found out that if the data masking policy on a particular table in Hive is on, data cannot be inserted. However, we can make the policy enabled once the data insert is complete.

Can you please help for guiding us on how can the Ranger Data Masking policies be disabled and enabled using an Unix command so that we can include those in the data ingestion workflow so that there ain't any manual intervention?

Looking for your guidance.

Thanks and Regards,

Rajdip

Re: Enabling and Disabling a Ranger Policy using command

jsensharma — Thu, 06 Apr 2017 22:56:21 GMT

@rajdip chaudhuri

You can use Ranger Rest API to be invoked using simple "curl" commands in order to disable/enable various ranger policies.

1. Get the List of Policies using Rest API: http://localhost:6080/service/plugins/policies For example you want to disable the policy ID = 14 (Example "HDFS Global Allow") The do the following:

Using Ranger API get the policy data:

curl -i -u admin:admin -H "Content-Type: application/json" -X GET http://localhost:6080/service/plugins/policies/14

2. Then you get some JSON output like following:

{"id":14,"guid":"eb167192-29a5-490c-9655-8be40c2363eb","isEnabled":true,"createdBy":"Admin","updatedBy":"Admin","createTime":1473764849000,"updateTime":1473764849000,"version":1,"service":"Sandbox_hadoop","name":"HDFS Global Allow","policyType":0,"description":"This policy gives global permission for all users. Disable this to test Apache Ranger","resourceSignature":"6be1f1907223a25472365fea64a3d450","isAuditEnabled":true,"resources":{"path":{"values":["/","/*"],"isExcludes":false,"isRecursive":true}},"policyItems":[{"accesses":[{"type":"read","isAllowed":true},{"type":"write","isAllowed":true},{"type":"execute","isAllowed":true}],"users":[],"groups":["public"],"conditions":[],"delegateAdmin":true}],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[],"rowFilterPolicyItems":[]}

Notice it has ["isEnabled":true]

3. Now Paste the output to a file "/tmp/14.json", then change it to ["isEnabled":false] and then PUT the data using curl. Content of "14.json" file as following:

{"id":14,"guid":"eb167192-29a5-490c-9655-8be40c2363eb","isEnabled":false,"createdBy":"Admin","updatedBy":"Admin","createTime":1473764849000,"updateTime":1473764849000,"version":1,"service":"Sandbox_hadoop","name":"HDFS Global Allow","policyType":0,"description":"This policy gives global permission for all users. Disable this to test Apache Ranger","resourceSignature":"6be1f1907223a25472365fea64a3d450","isAuditEnabled":true,"resources":{"path":{"values":["/","/*"],"isExcludes":false,"isRecursive":true}},"policyItems":[{"accesses":[{"type":"read","isAllowed":true},{"type":"write","isAllowed":true},{"type":"execute","isAllowed":true}],"users":[],"groups":["public"],"conditions":[],"delegateAdmin":true}],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[],"rowFilterPolicyItems":[]}

4. Now run the following curl command:

curl -i -u admin:admin -H "Content-Type: application/json" -X PUT -d@/tmp/14.json  http://localhost:6080/service/plugins/policies/14

More details: https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Policy+Management

Re: Enabling and Disabling a Ranger Policy using command

akulkarni1 — Fri, 07 Apr 2017 01:30:37 GMT

@rajdip chaudhuri

To add to @Jay SenSharma's answer: Please use a newer and better version of REST APIs as documented here.

Re: Enabling and Disabling a Ranger Policy using command

rajdip_chaudhur — Fri, 07 Apr 2017 14:01:23 GMT

Hello,

Am not sure if I am missing any steps here but while executing am getting error. Have followed @Jay SenSharma comments and able to get the json output and updated the flag in json. But while uploading the JSON after changes using PUT am facing error and it is not working.

Note that the RANGER is up and can perform operations from UI, but REST API PUT is not working (may be my error). Also the ip mentioned in below command contains the RANGER service.

Need you help as we are stuck here and everytime we had to do work manually which we want to bypass.

CURL command used to PUT the changed JSON is :

curl -i -u admin:admin -H "Content-Type: application/json" -X PUT -d@/tmp/10_2.json http://xx.xx.xx.207:6080/service/plugins/policies/10

(changed the ip)

Error thrown:

HTTP/1.1 404 Not Found Server: Apache-Coyote/1.1 Set-Cookie: RANGERADMINSESSIONID=03A8D6199168A17D4C19D442E8C55617; Path=/; HttpOnly X-Frame-Options: DENY Content-Length: 0 Date: Fri, 07 Apr 2017 06:56:50 GMT

Modified JSON:

HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Set-Cookie: RANGERADMINSESSIONID=EDCBDAFF124C9802A79BFD945662BC1A; Path=/; HttpOnly X-Frame-Options: DENY Content-Type: application/json Transfer-Encoding: chunked Date: Fri, 07 Apr 2017 07:00:49 GMT {"id":10,"guid":"c8afaae2-a4cc-4c25-b4b2-75ae9b0227eb","isEnabled":false,"createdBy":"Admin","updatedBy":"Admin","createTime":1491448221000,"updateTime":1491448221000,"version":1,"service":"TCSGEINTERNALCLUSTER_hive","name":"tcs_ge_user data masking test 2","policyType":1,"description":"tcs_ge_user data masking test 2","resourceSignature":"2cb6661609e66abfd9fbceaeac2be9d0","isAuditEnabled":true,"resources":{"database":{"values":["wells_fargo_poc"],"isExcludes":false,"isRecursive":false},"column":{"values":["card_number"],"isExcludes":false,"isRecursive":false},"table":{"values":["test_masked_2"],"isExcludes":false,"isRecursive":false}},"policyItems":[],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[{"accesses":[{"type":"select","isAllowed":true}],"users":["tcs_ge_user"],"groups":["tcs_ge_user"],"conditions":[],"delegateAdmin":false,"dataMaskInfo":{"dataMaskType":"MASK_HASH"}}],"rowFilterPolicyItems":[]}

Re: Enabling and Disabling a Ranger Policy using command

jsensharma — Fri, 07 Apr 2017 14:09:54 GMT

@rajdip chaudhuri

I am suspecting that your JSON file "/tmp/10_2.json" has the following line as well in it which is not right ... you should remove it.

HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Set-Cookie: RANGERADMINSESSIONID=EDCBDAFF124C9802A79BFD945662BC1A; Path=/; HttpOnly X-Frame-Options: DENY Content-Type: application/json Transfer-Encoding: chunked Date: Fri, 07 Apr 2017 07:00:49 GMT

Your modified JSON file "/tmp/10_2.json" should contain only the JSON data part. Looks like you have some additional data in it. (No other extra lines).

{"id":10,"guid":"c8afaae2-a4cc-4c25-b4b2-75ae9b0227eb","isEnabled":false,"createdBy":"Admin","updatedBy":"Admin","createTime":1491448221000,"updateTime":1491448221000,"version":1,"service":"TCSGEINTERNALCLUSTER_hive","name":"tcs_ge_user data masking test 2","policyType":1,"description":"tcs_ge_user data masking test 2","resourceSignature":"2cb6661609e66abfd9fbceaeac2be9d0","isAuditEnabled":true,"resources":{"database":{"values":["wells_fargo_poc"],"isExcludes":false,"isRecursive":false},"column":{"values":["card_number"],"isExcludes":false,"isRecursive":false},"table":{"values":["test_masked_2"],"isExcludes":false,"isRecursive":false}},"policyItems":[],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[{"accesses":[{"type":"select","isAllowed":true}],"users":["tcs_ge_user"],"groups":["tcs_ge_user"],"conditions":[],"delegateAdmin":false,"dataMaskInfo":{"dataMaskType":"MASK_HASH"}}],"rowFilterPolicyItems":[]}

Re: Enabling and Disabling a Ranger Policy using command

rajdip_chaudhur — Fri, 07 Apr 2017 14:13:30 GMT

Yes, you are absolutely correct. Can this part be removed somehow? Actually we are using "sed" to change the flag to false as we are trying to automate the whole process. If you can guide for removing that part then it will be of great help.

Thanks and Rajdip,

Rajdip

Re: Enabling and Disabling a Ranger Policy using command

jsensharma — Fri, 07 Apr 2017 14:21:59 GMT

@rajdip chaudhuri

In that case please do not use "-i" option in your curl GET command and redirect the output to a file as following using "-o" option:

curl  -u admin:admin -H "Content-Type: application/json" -X GET http://xx.xx.xx.207:6080/service/plugins/policies/10  -o /tmp/10_2.json

So that you only get the desired data not the response metadata.

Re: Enabling and Disabling a Ranger Policy using command

rajdip_chaudhur — Fri, 07 Apr 2017 15:01:47 GMT

It worked. Thanks a lot. Have also accepted the best answer.