I have a cluster running HDP 2.4.2 which has been running well for more than a year. The cluster is Kerberised and external applications connect via Knox. Earlier today, the cluster stopped accepting JDBC connections via Knox to Hive.
The Knox logs show no errors, but the Hive Server2 log shows the following error:
by: org.apache.hadoop.security.authorize.AuthorizationException: User:
knox is not allowed to impersonate <userid>
org.apache.hive.service.cli.HiveSQLException: Failed to validate proxy
privilege of knox for <userid>"
Having looked at other users the suggestions mostly seem to be around the correct setting of configuration options for hadoop.proxyusers.users and hadoop.proxyusers.groups. However, in my case I don't see how these settings could be the problem. The cluster has been running for over a year and we have a number of applications connecting to Hive via JDBC on a daily basis. The configuration of the server has not been changed and connections were previously succeeding on the current configuration. No changes had been made to the platform or environment and the cluster was not restarted or taken down for maintenance between the last successful JDBC connection and JDBC connections being declined.
I have now stopped and started the cluster, but after restart the cluster still does not accept JDBC connections.
Does anyone have any suggestions on how I should proceed?
If nothing has changed in the configuration then I would assume that something about the users have changed relative to the configuration. For instance, users may no longer be members of the "users" group or the Knox hosts have moved to different IP addresses than are configured.
@Imccay I don't see any evidence that the IP addresses have changed. Other access controlled via Knox is still working as expected e.g. we access Ambari as well as components on an HDF cluster via the same knox gateway and this is all working normally.
I'm not quite sure how to check membership of the users group or how group membership might have changed. The problem impacts a wide range of users. We have checked that users are still members of the same AD groups, both on our AD server and in Ranger.