Created 08-21-2018 05:05 AM
I was wondering if Apache Ranger (Including Ranger KMS) in HDP 2.6.4 fully supports WASB and ADLS. I would like to understand if we move to use WASB/ADLS instead of HDFS what functionalities will be impacted regarding Authentication, Authorisation and Auditing.
Created 08-21-2018 07:59 AM
Authorization params impacted: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/wasb-authoriza...
Authentication params impacted: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/authentication...
Auditing param impacted:
set these values in the ranger-hdfs-audit.xml , for Audit to HDFS on WASB
<property> <name>xasecure.audit.destination.hdfs</name> <value>enabled</value> </property> <property> <name>xasecure.audit.destination.hdfs.dir</name> <value>wasb://ranger-audit1@youraccount.blob.core.windows.net</value> </property> <property> <name>xasecure.audit.destination.hdfs.config.fs.azure.account.key.youraccount.blob.core.windows.net</name> <value>YOUR ENCRYPTED ACCESS KEY</value> </property> <property> <name>xasecure.audit.destination.hdfs.config.fs.azure.account.keyprovider.youraccount.blob.core.windows.net</name> <value>org.apache.hadoop.fs.azure.ShellDecryptionKeyProvider</value> </property> <property> <name>xasecure.audit.destination.hdfs.config.fs.azure.shellkeyprovider.script</name> <value>/usr/lib/python2.7/dist-packages/hdinsight_common/decrypt.sh</value> </property>
Note: Please upvote or accept the answer if you found it useful
Created 08-21-2018 10:24 AM
So is WASB fully supported by Ranger KMS to enable TDE?
Created 08-21-2018 03:49 PM
Hi ,
Apache Ranger 0.7.0 , shipped with HDP 2.6.4 supports WASB .
As per JIRA -> https://issues.apache.org/jira/browse/RANGER-1295
>While, ADLS does not seem to have support .
https://issues.apache.org/jira/browse/RANGER-1664.
2)
>Authorisation params impacted , please refer this
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/wasb-authoriza...
> Authentication params impacted, please refer this
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/authentication...
> For Auditing , you might want to set these values in the ranger-hdfs-audit.xml , for Audit to HDFS on WASB
===============================================
<property>
<name>xasecure.audit.destination.hdfs</name>
<value>enabled</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.dir</name>
<value>wasb://ranger-audit1@youraccount.blob.core.windows.net</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.config.fs.azure.account.key.youraccount.blob.core.windows.net</name>
<value>YOUR ENCRYPTED ACCESS KEY</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.config.fs.azure.account.keyprovider.youraccount.blob.core.windows.net</name>
<value>org.apache.hadoop.fs.azure.ShellDecryptionKeyProvider</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.config.fs.azure.shellkeyprovider.script</name>
<value>/usr/lib/python2.7/dist-packages/hdinsight_common/decrypt.sh</value>
</property>