Member since
03-04-2016
165
Posts
35
Kudos Received
7
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1687 | 06-20-2017 03:08 PM | |
8810 | 05-11-2017 09:59 AM | |
8889 | 01-12-2017 01:50 PM | |
1286 | 10-26-2016 03:02 PM | |
5271 | 09-06-2016 07:40 AM |
03-29-2017
02:57 PM
@vperiasamy I had but I deleted second Ranger Admin long time ago. Now I have single Ranger Admin server.
... View more
03-29-2017
02:56 PM
@Deepak Sharma it was only list of principles. My hadoop.security.auth_to_local is: RULE:[1:$1@$0](.*@EXAMPLE.COM)s/@.*//
RULE:[2:$1@$0](amshbase@EXAMPLE.COM)s/.*/ams/
RULE:[2:$1@$0](amshbase@EXAMPLE.COM)s/.*/hbase/
RULE:[2:$1@$0](amszk@EXAMPLE.COM)s/.*/ams/
RULE:[2:$1@$0](atlas@EXAMPLE.COM)s/.*/atlas/
RULE:[2:$1@$0](dn@EXAMPLE.COM)s/.*/hdfs/
RULE:[2:$1@$0](falcon@EXAMPLE.COM)s/.*/falcon/
RULE:[2:$1@$0](hbase@EXAMPLE.COM)s/.*/hbase/
RULE:[2:$1@$0](hive@EXAMPLE.COM)s/.*/hive/
RULE:[2:$1@$0](jhs@EXAMPLE.COM)s/.*/mapred/
RULE:[2:$1@$0](jn@EXAMPLE.COM)s/.*/hdfs/
RULE:[2:$1@$0](knox@EXAMPLE.COM)s/.*/knox/
RULE:[2:$1@$0](livy@EXAMPLE.COM)s/.*/livy/
RULE:[2:$1@$0](nm@EXAMPLE.COM)s/.*/yarn/
RULE:[2:$1@$0](nn@EXAMPLE.COM)s/.*/hdfs/
RULE:[2:$1@$0](oozie@EXAMPLE.COM)s/.*/oozie/
RULE:[2:$1@$0](rangeradmin@EXAMPLE.COM)s/.*/ranger/
RULE:[2:$1@$0](rangerkms@EXAMPLE.COM)s/.*/keyadmin/
RULE:[2:$1@$0](rangerusersync@EXAMPLE.COM)s/.*/rangerusersync/
RULE:[2:$1@$0](rm@EXAMPLE.COM)s/.*/yarn/
RULE:[2:$1@$0](yarn@EXAMPLE.COM)s/.*/yarn/
DEFAULT
... View more
03-29-2017
12:57 PM
I Regenerated Keytabs once again and restarted all services and still not working, here my HDFS repo:
... View more
03-29-2017
12:31 PM
@Deepak Sharma Sorry I missed this comment, here is my config:
... View more
03-29-2017
12:13 PM
Ok, but what is the password for hdfs user? I changed the user and password as it was shown here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_Ranger_Install_Guide/content/hdfs_plugin_kerberos.html
... View more
03-29-2017
11:57 AM
I have exactly the same rules that you uploaded, both in hdfs and hdfs repo. I deleted my old repo and let Ambari create new one, and the newly created HDFS repo has correct configs and test connection is done successfully.
... View more
03-29-2017
11:20 AM
Yes, I regenerated keytabs and restarted services. I dont get it: The log: 2017-03-29 13:26:35,429 ERROR client.RangerAdminRESTClient (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(124)) - Error getting policies. secureMode=true, user=nn/hadoop1.locald@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=CLUSTER_hadoop
2017-03-29 13:26:35,429 ERROR util.PolicyRefresher (PolicyRefresher.java:loadPolicyfromPolicyAdmin(240)) - PolicyRefresher(serviceName=CLUSTER_hadoop): failed to refresh policies. Will continue to use last known version of policies (3)
java.lang.Exception: HTTP 401
says user nn/hadoop1.locald@EXAMPLE.COM us unauthorized (HTTP 401), but below is: 2017-03-29 13:26:38,877 INFO ipc.Server (Server.java:saslProcess(1538)) - Auth successful for nn/hadoop1.locald@EXAMPLE.COM (auth:KERBEROS)
... View more
03-29-2017
10:47 AM
@Deepak Sharma thank you for a quick answer. Ranger is also Kerberized. I added those properties and changed Authentication Type in HDFS Repo to Kerberos. Now Test connection is done successfully, but the same error appears. After these changes few INFO logs appeared: 2017-03-29 12:46:23,368 ERROR client.RangerAdminRESTClient (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(124)) - Error getting policies. secureMode=true, user=nn/hadoop1.locald@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=3SOFT_HDL_hadoop
2017-03-29 12:46:23,368 ERROR util.PolicyRefresher (PolicyRefresher.java:loadPolicyfromPolicyAdmin(240)) - PolicyRefresher(serviceName=3SOFT_HDL_hadoop): failed to refresh policies. Will continue to use last known version of policies (3)
java.lang.Exception: HTTP 401
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:126)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:217)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:185)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158)
2017-03-29 12:46:24,577 WARN protocol.ResponseProcessCookies (ResponseProcessCookies.java:processCookies(122)) - Cookie rejected [hadoop.auth="", version:0, domain:hadoop1.locald, path:/, expiry:Thu Jan 01 01:00:00 CET 1970] Domain attribute "hadoop1.locald" violates the Netscape cookie specification
2017-03-29 12:46:24,582 WARN protocol.ResponseProcessCookies (ResponseProcessCookies.java:processCookies(122)) - Cookie rejected [hadoop.auth=""u=nn&p=nn/hadoop1.locald@EXAMPLE.COM&t=kerberos&e=1490820384581&s=hi0THf8d5c4wUgzQbs/+W/PENPo="", version:0, domain:hadoop1.locald, path:/, expiry:Wed Mar 29 22:46:24 CEST 2017] Domain attribute "hadoop1.locald" violates the Netscape cookie specification
2017-03-29 12:46:25,229 INFO BlockStateChange (BlockManager.java:computeReplicationWorkForBlocks(1580)) - BLOCK* neededReplications = 0, pendingReplications = 0.
2017-03-29 12:46:27,578 WARN protocol.ResponseProcessCookies (ResponseProcessCookies.java:processCookies(122)) - Cookie rejected [hadoop.auth="", version:0, domain:hadoop1.locald, path:/, expiry:Thu Jan 01 01:00:00 CET 1970] Domain attribute "hadoop1.locald" violates the Netscape cookie specification
2017-03-29 12:46:27,582 WARN protocol.ResponseProcessCookies (ResponseProcessCookies.java:processCookies(122)) - Cookie rejected [hadoop.auth=""u=nn&p=nn/hadoop1.locald@EXAMPLE.COM&t=kerberos&e=1490820387581&s=S0zta5LH3SfBXFh0XoB3T5ldjsQ="", version:0, domain:hadoop1.locald, path:/, expiry:Wed Mar 29 22:46:27 CEST 2017] Domain attribute "hadoop1.locald" violates the Netscape cookie specification
2017-03-29 12:46:28,230 INFO BlockStateChange (BlockManager.java:computeReplicationWorkForBlocks(1580)) - BLOCK* neededReplications = 0, pendingReplications = 0.
2017-03-29 12:46:28,474 INFO ipc.Server (Server.java:saslProcess(1538)) - Auth successful for nn/hadoop1.locald@EXAMPLE.COM (auth:KERBEROS)
2017-03-29 12:46:28,475 INFO authorize.ServiceAuthorizationManager (ServiceAuthorizationManager.java:authorize(137)) - Authorization successful for nn/hadoop1.locald@EXAMPLE.COM (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol
... View more
03-29-2017
10:03 AM
1 Kudo
Hi guys, Ranger fails to refresh policies after implementing Kerberos. I implemented Kerberos with new local MIT KDC, and using Ambari Automated Setup. HDFS, Hive and HBase works fine with new authentication method, but there are errors in refreshing policies. Every service where Ranger plugin is enabled gives me error: 2017-03-29 11:24:52,657 ERROR client.RangerAdminRESTClient (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(124)) - Error getting policies. secureMode=true, user=nn/hadoop1.locald@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=CLUSTER_hadoop
2017-03-29 11:24:52,657 ERROR util.PolicyRefresher (PolicyRefresher.java:loadPolicyfromPolicyAdmin(240)) - PolicyRefresher(serviceName=CLUSTER_hadoop): failed to refresh policies. Will continue to use last known version of policies (3)
java.lang.Exception: HTTP 401
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:126)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:217)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:185)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158)
Thats for HDFS, for other services the user is different (hive etc.). I am using HDP 2.5 and Ambari 2.4.1. These users exist in Kerberos (klist): hive/hadoop1.locald@EXAMPLE.COM
hive/hadoop2.locald@EXAMPLE.COM
hive/hadoop3.locald@EXAMPLE.COM
hive/hadoop4.locald@EXAMPLE.COM
infra-solr/hadoop1.locald@EXAMPLE.COM
jhs/hadoop2.locald@EXAMPLE.COM
jn/hadoop1.locald@EXAMPLE.COM
jn/hadoop2.locald@EXAMPLE.COM
jn/hadoop3.locald@EXAMPLE.COM
kadmin/admin@EXAMPLE.COM
kadmin/changepw@EXAMPLE.COM
kadmin/hadoop1.locald@EXAMPLE.COM
kafka/hadoop1.locald@EXAMPLE.COM
knox/hadoop1.locald@EXAMPLE.COM
krbtgt/EXAMPLE.COM@EXAMPLE.COM
livy/hadoop1.locald@EXAMPLE.COM
livy/hadoop2.locald@EXAMPLE.COM
livy/hadoop4.locald@EXAMPLE.COM
nm/hadoop1.locald@EXAMPLE.COM
nm/hadoop2.locald@EXAMPLE.COM
nm/hadoop3.locald@EXAMPLE.COM
nm/hadoop4.locald@EXAMPLE.COM
nn/hadoop1.locald@EXAMPLE.COM
nn/hadoop2.locald@EXAMPLE.COM
... View more
Labels:
- Labels:
-
Apache Ranger
03-22-2017
11:06 AM
@Apoorv Pathak Hi, yes it is working now. I used config posted above by Roman Glova.
... View more