How Hadoop service level authorization is different from Ranger Authorization. Can they be used together?

[#1 · Accepted solution]

Hi there @Rahul Pathak So there are really 3 components worth discussing in this topic:

1) Service level authorisation

2) Knox for access to Hadoop Services

3) Ranger for RBAC (Role Based Access Control) security policies.

First of all, these can all be used together, they provide additional complementary security measures for Hadoop.

Service Level Authorisation deals with the most basic set of permissions, all Hadoop services let you define the users and groups who are authorized to make RPC call to that service. Only if the user making RPC call belongs to authorized service user/group, the RPC call will go through. Once someone is through however, there is no further check made.

Knox takes this up a level and exposes user/group based policies for access to Hadoop services (Hive, HDFS, Storm etc) in a far easier way, the policy is created and applied by Ranger and enacted by Knox, this is true perimeter security as users can therefore be denied before they are able to even connect to the Hadoop cluster.

Ranger then gives the final level of granularity, once someone is granted access to a particular service, you can then control at a very granular level which Hive databases, tables and table colums they have access to, HDFS paths and the level of access, Kafka queues and much much more. This gives you fine grain control over the exact data and services you wish your users to be granted access to.

Hope that helps.

[#2]

Adding to drussell's response: You can set up both, Ranger and SLA policies (though not necessary, Ranger alone should be enough). Ranger policies will take precedence over SLAs. In the event that a Ranger policy does not exist then local SLA will take effect.

This video (https://www.youtube.com/watch?v=uCZKrKo5ebQ) gives a nice explanation of the workings of security in HDP.

[#3]

Thanks a lot.

[#4]

Can you help me with working demo of enabling service level authorisation for yarn.

I followed the steps in https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ServiceLevelAuth.html#Enabl... but it is not working.

I can run yarn jobs from any user irrespective of the acl settings. I tried this in HDP 2.3.4.0 with Ambari 2.2.0

FYI, ranger plugin policies are working fine. I tried this with and without enabling ranger plugin.

However service level authorisation is working fine in case of apache hadoop.