Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

REST api URL to configure Ranger objects

Solved Go to solution
Highlighted

REST api URL to configure Ranger objects

Guru

The way I can automatize kerberization of HDP cluster (here) via curl, I'm wondering if there is a REST api interface to configure Ranger as well. Now, I can see that the Ranger 0.5.0 exposes REST interfaces for Service Definition, Service and Policy Management (link). I'm trying to understand how to make use of these interfaces via curl.

Analysis of the Ranger UI traffic on my 2.3.2 sandbox, tells me that the Ranger UI follows different grammar what is described in the document above. For example, to get service by id, the document instructs to use this request URL:

Request URL : service/public/api/service/{id}

but this doesn't work on sandbox. It fails with this output:

$ curl  -u admin:admin -i -X GET http://127.0.0.1:6080/service/public/api/service/1
HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=C5F2FCA5D32A0521EE4F516C845C9AAD; Path=/; HttpOnly
Content-Length: 0
Date: Wed, 20 Jan 2016 16:19:57 GMT

What works is:

$ curl  -u admin:admin -i -X GET http://127.0.0.1:6080/service/plugins/policies/service/1

To make the matter worse, the document doesn't give an example of how should a complete request URL look like. Maybe I'm using wrong URL totally.

Any help would be appreciated. Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: REST api URL to configure Ranger objects

@Vipin Rathor Great question :) I have implemented a script at one of my customer that is actually adding policies and hdfs directories automatically as soon as a new users joins an AD group, so here is the part about how to use the RestAPI of Ranger to add policies.

HDFS Policy

Template:

{
	"policyName": "name_of_policy",
	"resourceName": "/path1,/path2/blub",
	"description": "",
	"repositoryName": "",
	"repositoryType": "hdfs",
	"isEnabled": "true",
	"isRecursive": "true",
	"isAuditEnabled": "true",
	"permMapList": [{
		"groupList": ["somegroup"],
		"permList": ["Read","Execute", "Write", "Admin"]
	}]
}

Curl:

curl -iv -u <user>:<password> -d @<policy payload> -H "Content-Type: application/json" -X POST http://<RANGER-Host>:6080/service/public/api/policy/

Hive Policy

Template:

{
   "policyName":"name_of_policy",
   "databases":"db1,db2",
   "tables":"mytable,yourtable",
   "columns":"",
   "udfs":"",
   "description":"",
   "repositoryName":"",
   "repositoryType":"hive",
   "tableType":"Inclusion",
   "columnType":"Inclusion",
   "isEnabled":"true",
   "isAuditEnabled":"true",
   "permMapList": [{
		"groupList": ["somegroup"],
		"permList": ["Select"]
	}]
}

Curl:

curl -iv -u <user>:<password> -d @<policy payload> -H "Content-Type: application/json" -X POST http://<RANGER-Host>:6080/service/public/api/policy/

Getting Policies

I just tested the Rest API to get some of my policies from Ranger, it worked. Make sure the Policy ID is valid, otherwise you'll get a "Data not found" error.

Curl

curl -iv -u <user>:<password> -H "Content-type:application/json" -X GET http://horton01.example.com:6080/service/public/api/policy/2

Result:

{
   "id":2,
   "createDate":"2015-11-21T07:03:21Z",
   "updateDate":"2015-12-08T05:54:24Z",
   "owner":"Admin",
   "updatedBy":"Admin",
   "policyName":"Ranger_audits",
   "resourceName":"/apps/solr/ranger_audits",
   "description":"",
   "repositoryName":"bigdata_hadoop",
   "repositoryType":"hdfs",
   "permMapList":[
      {
         "userList":[
            "solr"
         ],
         "groupList":[
         ],
         "permList":[
            "Read",
            "Write",
            "Execute"
         ]
      }
   ],
   "isEnabled":true,
   "isRecursive":true,
   "isAuditEnabled":false,
   "version":"5",
   "replacePerm":false
}

Let me know if you have any questions

View solution in original post

12 REPLIES 12
Highlighted

Re: REST api URL to configure Ranger objects

@Vipin Rathor Great question :) I have implemented a script at one of my customer that is actually adding policies and hdfs directories automatically as soon as a new users joins an AD group, so here is the part about how to use the RestAPI of Ranger to add policies.

HDFS Policy

Template:

{
	"policyName": "name_of_policy",
	"resourceName": "/path1,/path2/blub",
	"description": "",
	"repositoryName": "",
	"repositoryType": "hdfs",
	"isEnabled": "true",
	"isRecursive": "true",
	"isAuditEnabled": "true",
	"permMapList": [{
		"groupList": ["somegroup"],
		"permList": ["Read","Execute", "Write", "Admin"]
	}]
}

Curl:

curl -iv -u <user>:<password> -d @<policy payload> -H "Content-Type: application/json" -X POST http://<RANGER-Host>:6080/service/public/api/policy/

Hive Policy

Template:

{
   "policyName":"name_of_policy",
   "databases":"db1,db2",
   "tables":"mytable,yourtable",
   "columns":"",
   "udfs":"",
   "description":"",
   "repositoryName":"",
   "repositoryType":"hive",
   "tableType":"Inclusion",
   "columnType":"Inclusion",
   "isEnabled":"true",
   "isAuditEnabled":"true",
   "permMapList": [{
		"groupList": ["somegroup"],
		"permList": ["Select"]
	}]
}

Curl:

curl -iv -u <user>:<password> -d @<policy payload> -H "Content-Type: application/json" -X POST http://<RANGER-Host>:6080/service/public/api/policy/

Getting Policies

I just tested the Rest API to get some of my policies from Ranger, it worked. Make sure the Policy ID is valid, otherwise you'll get a "Data not found" error.

Curl

curl -iv -u <user>:<password> -H "Content-type:application/json" -X GET http://horton01.example.com:6080/service/public/api/policy/2

Result:

{
   "id":2,
   "createDate":"2015-11-21T07:03:21Z",
   "updateDate":"2015-12-08T05:54:24Z",
   "owner":"Admin",
   "updatedBy":"Admin",
   "policyName":"Ranger_audits",
   "resourceName":"/apps/solr/ranger_audits",
   "description":"",
   "repositoryName":"bigdata_hadoop",
   "repositoryType":"hdfs",
   "permMapList":[
      {
         "userList":[
            "solr"
         ],
         "groupList":[
         ],
         "permList":[
            "Read",
            "Write",
            "Execute"
         ]
      }
   ],
   "isEnabled":true,
   "isRecursive":true,
   "isAuditEnabled":false,
   "version":"5",
   "replacePerm":false
}

Let me know if you have any questions

View solution in original post

Highlighted

Re: REST api URL to configure Ranger objects

Guru

Thanks a lot @Jonas Straub !

This is valuable and very helpful input. Cheers.

Highlighted

Re: REST api URL to configure Ranger objects

Expert Contributor

@Jonas Straub This is indeed useful.

I am able export the policy repositories and policies (by policyId) successfully.

However I noticed that exported policy (json) contains ids, now I am trying to create the same policy on the target cluster. However, I do not see the same Id in target db.

Mover, I noticed that on source and target ranger db, the service names (i.e. repository names are different), manually I had to change it. But exporting from cluster1 and applying policy on cluster2 worked fine for me.

Thank you.

Referred documentation here.

Highlighted

Re: REST api URL to configure Ranger objects

Expert Contributor

@Jonas Straub, @Vipin Rathor : Using this I could export one policy at a time, Is there a way to export all the policies at the same time downloaded using

http://<ranger_address>:6080/service/plugins/policies/download/<clustername>_hadoop
Highlighted

Re: REST api URL to configure Ranger objects

Cloudera Employee

Thanks a lot @Jonas Straub

This is amazing and helped save me a lot of time. Appreciate it! :)

Highlighted

Re: REST api URL to configure Ranger objects

Explorer

Hello All,

Whenever I try to create HDFS policy using @Jonas Straub script i keep getting an error the READ is not a valid access type. any clue what might be the problem?

{"statusCode":1,"msgDesc":"Read: is not a valid access-type. policy='Test' service='HAD_hadoop'"}

Highlighted

Re: REST api URL to configure Ranger objects

Explorer

UPDATE: i was able to figure out what the issue was. In Ranger 0.5, Permlist values are supposed to be all lower case "read, write, execute"

Highlighted

Re: REST api URL to configure Ranger objects

Interesting. Thanks for letting us know :)

Highlighted

Re: REST api URL to configure Ranger objects

Explorer

Hi Vipin, Can you share us the script for adding policies and hdfs directories automatically as soon as a new users joins an AD group.

Don't have an account?
Coming from Hortonworks? Activate your account here