WebHDFS: KMS Cache Issue with HDFS TDE

[#1]

In previous releases of HDP, client-side caching of keys could result in unexpected behavior with WebHDFS.

Consider the following steps:

1. Create two keys in ranger KMS: user1_key and user2_key

2. Add two resource based policy one per above user.

User1_encr_policy: Allow the Decrypt_EEK permissions to user1 only

User2_encr_policy: Allow the Decrypt_EEK permissions to user2 only.

3. Add two encryption zones. user1_zone (using user1_key) and user2_zone (using user2_key)

4. Run the following command, you may be able to access the content of test.csv file from user1_zone using user2

curl -i -L "http://sandbox.hortonworks.com:50070/webhdfs/v1/customer/user1_zone/test.csv?user.name=user2&op=OPEN"

HDP-2.6.1.2 includes HADOOP-13749, which fixes the caching issue. The FS cache and KMS provider cache can be disabled by changing the configuration as follows: "fs.hdfs.impl.disable.cache", "true" dfs.client.key.provider.cache.expiry, 0