Member since
04-20-2023
2
Posts
0
Kudos Received
0
Solutions
04-20-2023
09:09 AM
PS. It seems I made some progress. I consulted this documentation https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/authorization-ranger/sec_authorization_ranger.pdf and made sure to include the rules from the hadoop.security.auth_to_local property as found in /etc/hadoop/conf. I can now test the connection in the HDFS service in Ranger and that tests ok. The error I get now though is 2023-04-20 18:05:27,074 ERROR util.PolicyRefresher (PolicyRefresher.java:loadPolicyfromPolicyAdmin(275)) - PolicyRefresher(serviceName=XYZ_hadoop): failed to refresh policies. Will continue to use last known version of policies (-1) com.sun.jersey.api.client.ClientHandlerException: java.lang.RuntimeException: java.lang.NullPointerException at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) at com.sun.jersey.api.client.Client.handle(Client.java:652) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) at org.apache.ranger.admin.client.RangerAdminRESTClient$3.run(RangerAdminRESTClient.java:140) at org.apache.ranger.admin.client.RangerAdminRESTClient$3.run(RangerAdminRESTClient.java:132) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:143) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:251) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191) at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:161) Caused by: java.lang.RuntimeException: java.lang.NullPointerException at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1488) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3018) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:489) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ... 13 more Caused by: java.lang.NullPointerException at java.util.Base64$Encoder.encode(Base64.java:261) at java.util.Base64$Encoder.encodeToString(Base64.java:315) at sun.net.www.protocol.http.NegotiateAuthentication.setHeaders(NegotiateAuthentication.java:182) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1731) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ... 15 more So it seems the NameNode still is not able to obtain the relevant information from Ranger.
... View more
04-20-2023
08:50 AM
Hi, I am running an HDP 3.1.4 cluster with Kerberos and Ranger enabled. When setting up the cluster, a bunch of default services and policies were created such as XYZ_yarn, XYZ_hive, XYZ_hbase, where XYZ is my cluster's name. It seems there should a policy XYZ_hadoop for the HDFS service; on every HDFS access, I get an exception such as "org.apache.ranger.plugin.util.RangerServiceNotFoundException: XYZ_hadoop" in the NameNode logs. I am not sure when the error started or if I ever had such a policy in place; the problem persisted as far back as my namenode logs are available. My question now is how to (re-)create the XYZ_hadoop policy; for reasons of symmetry, I would assume Ambari (?) created it when I set up the cluster. Mainly, I don't know what to put into the Username and Password fields - the "hdfs" user on my nodes doesn't have a password as per /etc/shadow, and on the Kerberos side, as far as I understand everything works with keytabs, and I don't know which principal and which password I would put in the service form. The Cloudera docs say to put "the end system username that can be used for connection", but I have now idea what that means, particularly in a Kerberized cluster. Thanks for any kind of input :-)!
... View more
Labels:
- Labels:
-
Apache Ranger