HDP to CDP - Ranger policies export and import

[#1]

Introduction 

Apache Ranger delivers a comprehensive approach to security for a Hadoop cluster. It provides a centralised platform to define, administer and manage security policies consistently across Hadoop components.

More details about Ranger can be found here 

https://www.cloudera.com/products/open-source/apache-hadoop/apache-ranger.html 

Ranger API policy documentation https://ranger.apache.org/apidocs/index.html 

This article focuses on Export and Import of Ranger policies using API calls during HDP to CDP migration. 

Export 

• List of Services configured in Ranger 

### Command to get list of services 

curl -s -u admin:pass -H "Accept: application/json" -H "Content-Type: application/json" -X GET "http://<hostname>:<ranger-port>/service/public/v2/api/service" > services.json

• Export of Policies 

### Export all policies 

To export all policies
curl -X GET --header "text/json" -H "Content-Type: text/json" -o file.json -u admin:admin "http://<hostname>:<ranger-port>/service/plugins/policies/exportJson"

 This exported json file.json contains all policies including Tag based policies  

• Export of users and Groups, which can be used for validation purposes.

## Api call to download all Users from Ranger 
curl -s -u admin:pass -H "Accept: application/json" -H "Content-Type: application/json" -X GET "https://ranger.com/service/xusers/users" > users.json

## Api call to download all groups from Ranger 
curl -s -u admin:pass -H "Accept: application/json" -H "Content-Type: application/json" -X GET "https://ranger.com/service/xusers/groups" > groups.json

Import

Importing policies into Target CDP cluster

• Step 1: Prepare the Ranger service and make sure to configure all service plugins.
• Step 2: Prepare servicemapping.json file which has mapping of Ranger service from HDP to CDP world

cat /path/servicesMapping.json

{"cm_knox":"cm_knox","cm_hdfs":"cm_hdfs","cm_hbase":"cm_hbase","cm_yarn":"cm_yarn","cm_solr":"cm_solr","cm_kafka":"cm_kafka","cm_atlas":"cm_atlas","cm_hive":"cm_hive"}

• Step 3: Import the Ranger policies using Ranger API

#To Import policies from JSON file with servicesMap

curl -i -X POST -H "Content-Type: multipart/form-data" -F 'file=@/path/file.json'  -F ‘servicesMapJson=@/path/servicesMapping.json’ -u admin:admin http://<hostname>:<ranger-port>/service/plugins/policies/importPoliciesFromFile?isOverride=true

Preparation for HDP to CDP Migration

Known threats and Todo's

• Local users/groups in HDP Ranger must be available in Target CDP cluster.
• AD/LDAP users/groups in HDP Ranger must be available in Target CDP cluster.
• Ranger Services in HDP cluster must be configured in CDP clusters.
• Before importing policies into CDP Ranger must be empty ( Make sure to delete default policies which we get during enabling of services)
• Default policies must be reviewed and cleaned (e.g public groups and all resource are not ideal for production clusters)

Useful Links 

• Review and add Ranger policies required in CDP world which can be found here. https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/upgrade-hdp3/topics/amb3-add-ranger-polic...