Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
New Contributor

Apache Ranger delivers a comprehensive approach to security for a Hadoop cluster. It provides central security policy administration across the core enterprise security requirements of authorization, accounting and data protection.

Apache Ranger already extends baseline features for coordinated enforcement across Hadoop workloads from batch, interactive SQL and real–time in Hadoop.

In this tutorial, we cover using Apache Ranger for HDP 2.3 to secure your Hadoop environment. We will walkthrough the following topics:

  1. Support for Knox authorization and audit
  2. Command line policies in Hive
  3. Command line policies in HBase
  4. REST APIs for policy manager

Prerequisite

The only prerequisite for this tutorial is that you have Hortonworks Sandbox.

Once you have Hortonworks Sandbox, login through SSH:

Starting Knox Service and Demo LDAP Service

From the Ambari console at http://localhost:8080/ (username and password is admin and adminrespectively), select Knox from the list of Services on the left-hand side of the page.

Then click on Service Actions from the top right hand side of the page click on Start

![]http://www.dropbox.com/s/jhb30dgey8m30n6/Screenshot%202015-09-08%2010.27.06.png?dl=1)

From the following you can track the start of the Knox service to completion:

Then go back to the Service Actions button on the Knox service and click on Start Demo LDAP

You can track the start of the Demo LDAP Service from the following screen:

Knox access scenarios

Check if Ranger Admin console is running, at http://localhost:6080/from your host machine. The username is admin and the password is admin

If it is not running you can start from the command line using the command

sudo service ranger-admin start

Click on sandbox_knox link under Knox section in the main screen of Ranger Administration Portal

You can review policy details by a clicking on the policy name.

To start testing Knox policies, we would need to turn off the “global knox allow” policy.

Locate Sandbox for Guest policy on the Ranger Admin console and edit the policy

and enable policy named “Sandbox for Guest”

From your local SSHd terminal (not directly on the Sandbox), run this CURL command to access WebHDFS

curl -k -u admin:admin-password 'https://127.0.0.1:8443/gateway/knox_sample/webhdfs/v1?op=LISTSTATUS'

Go to Ranger Policy Manager tool → Audit screen and check the knox access (denied) being audited.

Now let us try the same CURL command using “guest” user credentials from the terminal

curl -k -u guest:guest-password 'https://127.0.0.1:8443/gateway/knox_sample/webhdfs/v1?op=LISTSTATUS'

<code>{"FileStatuses":{"FileStatus":[{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":16393,"group":"hadoop","length":0,"modificationTime":1439987528048,"owner":"yarn","pathSuffix":"app-logs","permission":"777","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":4,"fileId":16389,"group":"hdfs","length":0,"modificationTime":1439987809562,"owner":"hdfs","pathSuffix":"apps","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":17000,"group":"hdfs","length":0,"modificationTime":1439989173392,"owner":"hdfs","pathSuffix":"demo","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16398,"group":"hdfs","length":0,"modificationTime":1439987529660,"owner":"hdfs","pathSuffix":"hdp","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16394,"group":"hdfs","length":0,"modificationTime":1439987528532,"owner":"mapred","pathSuffix":"mapred","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":2,"fileId":16396,"group":"hadoop","length":0,"modificationTime":1439987538099,"owner":"mapred","pathSuffix":"mr-history","permission":"777","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16954,"group":"hdfs","length":0,"modificationTime":1439988741413,"owner":"hdfs","pathSuffix":"ranger","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":16386,"group":"hdfs","length":0,"modificationTime":1440165443820,"owner":"hdfs","pathSuffix":"tmp","permission":"777","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":8,"fileId":16387,"group":"hdfs","length":0,"modificationTime":1439988397561,"owner":"hdfs","pathSuffix":"user","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}]}}

We can check the auditing in the Ranger Policy Manager → Audit screen

Ranger plugin for Knox intercepts any request made to Knox and enforces policies which are retrieved from the Ranger Administration Portal

You can configure the Knox policies in Ranger to restrict to a specific service (WebHDFS, WebHCAT etc) and to a specific user or a group and you can even bind user/group to an ip address

Hive grant/revoke permission scenarios

Ranger can support import of grant/revoke policies set through command line or Hue for Hive. Ranger can store these policies centrally along with policies created in the administration portal and enforce it in Hive using its plugin.

As a first step, disable the global access policy for Hive in Ranger Administration Portal

Let us try running a Grant operation using user Hive from the command line. Login into beeline tool using the following command

beeline -u "jdbc:hive2://sandbox.hortonworks.com:10000/default" -n it1 -p it1-d org.apache.hive.jdbc.HiveDriver

Then issue the GRANT command

grant select, update on table xademo.customer_details to user network1;

You should see the following error:

Let us check the audit log in the Ranger Administration Portal → Audit

You can see that access was denied for an admin operation for user it1.

We can create a policy in Ranger for user ‘it1’ to be an admin. Create a new policy from the Ranger Admin console and ensure the configuration matches the illustration below

We can try the beeline command again, once the policy has been saved.

GRANT select, update on table xademo.customer_details to user network1;

If the command goes through successfully, you will see the policy created/updated in Ranger Admin Portal → Policy Manager. It checks if there is an existing relevant policy to update, else it creates a new one.

What happened here?

Ranger plugin intercepts GRANT/REVOKE commands in Hive and creates corresponding policies in Admin portal. The plugin then uses these policies for enforcing Hive authorization (Hiveserver2)

Users can run further GRANT commands to update permissions and REVOKE commands to take away permissions.

HBase grant/revoke permission scenarios

Ranger can support import of grant/revoke policies set through command line in Hbase. Similar to Hive, Ranger can store these policies as part of the Policy Manager and enforce it in Hbase using its plugin.

Before you go further, ensure HBase is running from Ambari – http://127.0.0.1:8080 (username and password are admin).

If it is not go to Service Actions button on top right and Start the service

As a first step, let us try running a Grant operation using user Hbase.

Disable the public access policy “HBase Global Allow” in Ranger Administration Portal – policy manager

Login into HBase shell as ‘it1’ user

<code>su - it1

[it1@sandbox ~]$ hbase shell

Run a grant command to give “Read”, “Write”, “Create” access to user mktg1 in table ‘iemployee’

hbase(main):001:0> grant 'mktg1', 'RWC', 'iemployee'

you should get a Acess Denied as below:

Go to Ranger Administration Portal → Policy Manager and create a new policy to assign “admin” rights to user it1

Save the policy and rerun the HBase command again

<code>hbase(main):006:0> grant 'mktg1', 'RWC', 'iemployee'

0 row(s) in 0.8670 seconds

Check HBase policies in the Ranger Policy Administration portal. The grant permissions were added to an existing policy for table ‘iemployee’ that we created in previous step

You can revoke the same permissions and the permissions will be removed from Ranger admin. Try this in the same HBase shell

<code>hbase(main):007:0> revoke 'mktg1', 'iemployee'

0 row(s) in 0.4330 seconds

You can check the existing policy and see if it has been changed

What happened here?

Ranger plugin intercepts GRANT/REVOKE commands in Hbase and creates corresponding policies in the Admin portal. The plugin then uses these policies for enforcing authorization

Users can run further GRANT commands to update permissions and REVOKE commands to take away permissions.

REST APIs for Policy Administration

Ranger policies administration can be managed through REST APIs. Users can use the APIs to create or update policies, instead of going into the Administration Portal.

Running REST APIs from command line

From your local command line shell, run this CURL command. This API will create a policy with the name “hadoopdev-testing-policy2” within the HDFS repository “sandbox_hdfs”

<code>curl -i --header "Accept:application/json" -H "Content-Type: application/json" --user admin:admin -X POST http://127.0.0.1:6080/service/public/api/policy -d '{ "policyName":"hadoopdev-testing-policy2","resourceName":"/demo/data/test","description":"Testing policy for /demo/data/test","repositoryName":"sandbox_hdfs","repositoryType":"HDFS","permMapList":[{"userList":["mktg1"],"permList":["Read"]},{"groupList":["IT"],"permList":["Read"]}],"isEnabled":true,"isRecursive":true,"isAuditEnabled":true,"version":"0.1.0","replacePerm":false}'

the policy manager and see the new policy named “hadoopdev-testing-policy2”

Click on the policy and check the permissions that has been created

The policy id is part of the URL of this policy detail pagehttp://127.0.0.1:6080/index.html#!/hdfs/1/policy/26

We can use the policy id to retrieve or change the policy.

Run the below CURL command to get policy details using API

curl -i --user admin:admin -X GET http://127.0.0.1:6080/service/public/api/policy/26

What happened here?

We created a policy and retrieved policy details using REST APIs. Users can now manage their policies using API tools or applications integrated with the Ranger REST APIs

Hopefully, through this whirlwind tour of Ranger, you were introduced to the simplicity and power of Ranger for security administration.

2,516 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎09-25-2015 12:03 AM
Updated by:
 
Contributors
Top Kudoed Authors