HDFS not allowed to do 'GENERATE_EEK' on 'hive'

[#1]

I've recently upgraded the cluster to HDP 2.5.3 as well as Ambari to 2.4.2.0 however I'm now facing problems running Hive queries.

Each query that invokes Tez (i.e. `insert`) results in the following error:

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): java.util.concurrent.ExecutionException: org.apache.hadoop.security.authorize.Authori
zationException: User:hdfs not allowed to do 'GENERATE_EEK' on 'hive'

Here are my commands:

$ kinit -kt /etc/security/keytabs/automation.keytab
$ beeline -u 'jdbc:hive2://hiverserver2:10000/default;principal=hive/hiverserver2@ACTIVE.DIRECTORY' -f hive_script.hql

This is obviously something that was working before the upgrade.

Why is it running the script as the hdfs user? I have not added the `hdfs` user to the 'GENERATE_EEK' property on the Ranger KMS UI as this is not advised (and also not permitted).

Are there any settings that need to be adjusted after the upgrade?

[#2 · Accepted solution]

There was an issue with the Ranger KMS UI which prevented me from making any changes to the policy. Instead I used the API to update the policy which worked successfully.

The change I made was to add the HDFS user to the 'GENERATE_EEK' policy.

API documentation and resources:

https://community.hortonworks.com/articles/76118/how-to-access-ranger-kms-policies-via-rest-api.html

https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.6+-+REST+APIs+for+Service+Definit...