CDP 7.1.6 Ranger KMS test conection failed " User:ranger not allowed to do 'GET_KEYS' "

[#1]

Hello Gurus,

I am having Ranger KMS test connection failed, it is POC test.

CDP 7.1.6 with Isilon OneFS v8.2.2.0, AD kerberos enabled.

Ranger KMS is up and runningdefault policy login as keyadmintest connection failed

Already added following lines in kms-site.xml ( added in Ranger KMS -configuration )

hadoop.kms.proxyuser.rangeradmin.hosts=* 
hadoop.kms.proxyuser.rangeradmin.groups=* 
hadoop.kms.proxyuser.rangeradmin.users=*

Ranger KMS debug:

2021-06-26 06:51:38,420 DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader: ==> RangerPluginClassLoader.deactivate()
2021-06-26 06:51:38,420 DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader: <== RangerPluginClassLoader.deactivate()
2021-06-26 06:51:38,420 ERROR org.apache.hadoop.crypto.key.kms.server.KMS: Exception in getkeyNames.
org.apache.hadoop.security.authorize.AuthorizationException: User:ranger not allowed to do 'GET_KEYS'
2021-06-26 06:51:38,420 WARN org.apache.hadoop.crypto.key.kms.server.KMS: User ranger (auth:PROXY) via rangeradmin/n02.py.local@PY.LOCAL (auth:KERBEROS) request GET http://n03.py.local:9292/kms/v1/keys/names?doAs=ranger caused exception.
org.apache.hadoop.security.authorize.AuthorizationException: User:ranger not allowed to do 'GET_KEYS'
2021-06-26 06:52:04,559 INFO org.apache.ranger.audit.provider.BaseAuditHandler: Audit Status Log: name=kms.async.summary.multi_dest.batch.solr, interval=01:00.003 minutes, events=1, deferredCount=1, totalEvents=3, totalDeferredCount=3
2021-06-26 06:52:04,560 INFO org.apache.ranger.audit.destination.SolrAuditDestination: Solr zkHosts=null, solrURLs=null, collectionName=ranger_audits
2021-06-26 06:52:04,560 ERROR org.apache.ranger.audit.queue.AuditFileSpool: Error sending logs to consumer. provider=kms.async.summary.multi_dest.batch, consumer=kms.async.summary.multi_dest.batch.solr
2021-06-26 06:52:04,560 INFO org.apache.ranger.audit.queue.AuditFileSpool: Destination is down. sleeping for 30000 milli seconds. indexQueue=0, queueName=kms.async.summary.multi_dest.batch, consumer=kms.async.summary.multi_dest.batch.solr
2021-06-26 06:52:04,691 INFO org.apache.ranger.audit.provider.BaseAuditHandler: Audit Status Log: name=kms.async.summary.multi_dest.batch.hdfs, interval=01:00.012 minutes, events=1, deferredCount=1, totalEvents=3, totalDeferredCount=3

Is there anything mis-configured or need to be checked? Thank you

Best Regards,

Jake Zhang

[#2]

yeah, I was checking the KMS logs, not sure if there is something mis-configured....

2021-07-01 10:03:35,980 DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient: ==> RangerAdminRESTClient.getServicePoliciesIfUpdated(-1, 1625097035937)
2021-07-01 10:03:35,980 DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient: Checking Service policy if updated with old api call
2021-07-01 10:03:35,986 DEBUG org.apache.ranger.admin.client.datatype.RESTResponse: fromJson('Unauthenticated access not allowed') failed
org.codehaus.jackson.JsonParseException: Unexpected character ('U' (code 85)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: java.io.StringReader@7b831251; line: 1, column: 2]
        at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433)
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:521)
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:442)
        at org.codehaus.jackson.impl.ReaderBasedParser._handleUnexpectedValue(ReaderBasedParser.java:1198)
        at org.codehaus.jackson.impl.ReaderBasedParser.nextToken(ReaderBasedParser.java:485)
        at org.codehaus.jackson.map.ObjectMapper._initForReading(ObjectMapper.java:2770)
        at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2718)
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863)
        at org.apache.ranger.plugin.util.JsonUtilsV2.jsonToObj(JsonUtilsV2.java:68)
        at org.apache.ranger.admin.client.datatype.RESTResponse.fromJson(RESTResponse.java:126)
        at org.apache.ranger.admin.client.datatype.RESTResponse.fromClientResponse(RESTResponse.java:100)
        at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:195)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:305)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:244)
        at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:206)
2021-07-01 10:03:35,987 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=false, user=kms (auth:SIMPLE), response={"httpStatusCode":400,"statusCode":0}, serviceName=cm_kms
2021-07-01 10:03:35,987 DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient: <== RangerAdminRESTClient.getServicePoliciesIfUpdated(-1, 1625097035937): null
2021-07-01 10:03:35,987 DEBUG org.apache.ranger.plugin.util.PolicyRefresher: PolicyRefresher(serviceName=cm_kms).run(): no update found. lastKnownVersion=-1
2021-07-01 10:03:35,987 DEBUG org.apache.ranger.perf.policyengine.init: [PERF] PolicyRefresher.loadPolicyFromPolicyAdmin(serviceName=cm_kms): 7
2021-07-01 10:03:35,987 DEBUG org.apache.ranger.plugin.util.PolicyRefresher: <== PolicyRefresher(serviceName=cm_kms).loadPolicyfromPolicyAdmin()

Ranger KMS authenticatin type is kerberos, I tired to change it to simple and restarted both ranger and rangerkms, it did not help.

I don't know where the auth simple come from? Thanks.

2021-07-01 10:03:35,987 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=false, user=kms (auth:SIMPLE), response={"httpStatusCode":400,"statusCode":0}, serviceName=cm_kms

[#3]

I was seeing the same issue, thanks to @jakezhang  for posting.

Changing the Ranger KMS: kerberos_princ_name from rangerkms to keyadmin allowed me to get this working. Thanks for the clues in the log file and to @Scharan