Cloudera Community
leezy
user rank
Expert Contributor
My Accepted Solutions
Show All

Re: Atlas Metadata server start fail with hbase ta...

by Expert Contributor in Support Questions
‎03-01-2017 02:37 AM
‎03-01-2017 02:37 AM
thanks，i am sure i have add the same properties and use the hbase user. but i find other service policy update also not work. here is the ranger access log: 192.168.55.205 - - [01/Mar/2017:10:00:03 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseMaster@bigdata6-venus_bigdata_hbase HTTP/1.1" 401 - 192.168.55.205 - - [01/Mar/2017:10:00:03 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hive?lastKnownVersion=3&pluginId=hiveServer2@bigdata6-venus_bigdata_hive HTTP/1.1" 401 - 192.168.55.206 - - [01/Mar/2017:10:00:04 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseRegional@bigdata7-venus_bigdata_hbase HTTP/1.1" 401 - 192.168.55.207 - - [01/Mar/2017:10:00:04 +0800] "GET /service/plugins/policies/download/venus_bigdata_storm?lastKnownVersion=3&pluginId=storm@bigdata8-venus_bigdata_storm HTTP/1.1" 304 - 192.168.55.205 - - [01/Mar/2017:10:00:05 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hadoop?lastKnownVersion=2&pluginId=hdfs@bigdata6-venus_bigdata_hadoop HTTP/1.1" 401 - 192.168.55.206 - - [01/Mar/2017:10:00:05 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hadoop?lastKnownVersion=2&pluginId=hdfs@bigdata7-venus_bigdata_hadoop HTTP/1.1" 401 - 192.168.55.207 - - [01/Mar/2017:10:00:05 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseRegional@bigdata8-venus_bigdata_hbase HTTP/1.1" 401 - 192.168.55.205 - - [01/Mar/2017:10:00:05 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_yarn?lastKnownVersion=2&pluginId=yarn@bigdata6-venus_bigdata_yarn HTTP/1.1" 401 - 192.168.55.208 - - [01/Mar/2017:10:00:07 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseRegional@bigdata9-venus_bigdata_hbase HTTP/1.1" 401 - 192.168.55.205 - - [01/Mar/2017:10:00:08 +0800] "GET /login.jsp HTTP/1.1" 200 3325 192.168.55.207 - - [01/Mar/2017:10:00:11 +0800] "GET /service/plugins/policies/download/venus_bigdata_storm?lastKnownVersion=3&pluginId=storm@bigdata8-venus_bigdata_storm HTTP/1.1" 304 - 192.168.55.208 - - [01/Mar/2017:10:00:13 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_kafka?lastKnownVersion=4&pluginId=kafka@bigdata9-venus_bigdata_kafka HTTP/1.1" 401 - 192.168.55.206 - - [01/Mar/2017:10:00:13 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_kafka?lastKnownVersion=4&pluginId=kafka@bigdata7-venus_bigdata_kafka HTTP/1.1" 401 - 192.168.55.208 - - [01/Mar/2017:10:00:13 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_kafka?lastKnownVersion=4&pluginId=kafka@bigdata9-venus_bigdata_kafka HTTP/1.1" 304 - 192.168.55.206 - - [01/Mar/2017:10:00:13 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_kafka?lastKnownVersion=4&pluginId=kafka@bigdata7-venus_bigdata_kafka HTTP/1.1" 304 - 192.168.55.205 - - [01/Mar/2017:10:00:33 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseMaster@bigdata6-venus_bigdata_hbase HTTP/1.1" 401 - 192.168.55.205 - - [01/Mar/2017:10:00:33 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hive?lastKnownVersion=3&pluginId=hiveServer2@bigdata6-venus_bigdata_hive HTTP/1.1" 401 - ... View more

Re: Atlas Metadata server start fail with hbase ta...

by Expert Contributor in Support Questions
‎02-28-2017 10:03 AM
‎02-28-2017 10:03 AM
i use hdp 2.5.3.0 and the cluster has kerberised. And the properties you mentioned have added, but not work. ... View more

Re: Atlas Metadata server start fail with hbase ta...

by Expert Contributor in Support Questions
‎02-28-2017 09:09 AM
‎02-28-2017 09:09 AM
when i send the request use curl -i --negotiate -u hbase "http://bigdata6:6080/service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseMaster@bigdata6-venus_bigdata_hbase", i return the results, i don't know the way hbase plugin works. ... View more

Re: Atlas Metadata server start fail with hbase ta...

by Expert Contributor in Support Questions
‎02-28-2017 07:53 AM
‎02-28-2017 07:53 AM
thanks @Deepak Sharma. i find the problem in the log the hbase log: 2017-02-28 15:48:58,330 ERROR [Thread-74] client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=hbase/bigdata6@VENUS.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=venus_bigdata_hbase 2017-02-28 15:48:58,330 ERROR [Thread-74] util.PolicyRefresher: PolicyRefresher(serviceName=venus_bigdata_hbase): failed to refresh policies. Will continue to use last known version of policies (4) java.lang.Exception: HTTP 401 at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:126) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:232) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:188) at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158) and the ranger access log: 192.168.55.205 - - [28/Feb/2017:15:18:24 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseMaster@bigdata6-venus_bigdata_hbase HTTP/1.1" 401 - 192.168.55.206 - - [28/Feb/2017:15:18:24 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseRegional@bigdata7-venus_bigdata_hbase HTTP/1.1" 401 - 192.168.55.207 - - [28/Feb/2017:15:18:25 +0800] "GET /service/plugins/secure/policies/download/venus_bigdata_hbase?lastKnownVersion=4&pluginId=hbaseRegional@bigdata8-venus_bigdata_hbase HTTP/1.1" 401 ------------ the ranger audit page have no edit history recently,and the same to other service. i don't know how to solve it. Please give me some advice, thanks. ... View more

Re: Atlas Metadata server start fail with hbase ta...

by Expert Contributor in Support Questions
‎02-27-2017 01:42 AM
‎02-27-2017 01:42 AM
How to check? i‘ve restart ranger many times, and not work. ... View more

Atlas Metadata server start fail with hbase table ...

by Expert Contributor in Support Questions
‎02-24-2017 09:51 AM
2 Kudos
‎02-24-2017 09:51 AM
2 Kudos
Hi ,Atlas Metadata server start fail and i find the reason is the hbase table grant operation was denied by ranger. The doc has said that the permissions do not have the grant. I don't know why. the audit log and ranger policy: here is the log: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py", line 231, in <module> MetadataServer().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 720, in restart self.start(env, upgrade_type=upgrade_type) File "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py", line 92, in start user=params.hbase_user File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'kinit -kt /etc/security/keytabs/hbase.headless.keytab hbase-venus_bigdata@VENUS.COM; cat /var/lib/ambari-agent/tmp/atlas_hbase_setup.rb | hbase shell -n' returned 1. atlas_titan ATLAS_ENTITY_AUDIT_EVENTS atlas TABLE ATLAS_ENTITY_AUDIT_EVENTS access_tracker alertDataSource alertExecutor alertStream alertStreamSchema alertdef alertdetail atlas_titan eagle_metric eaglehdfs_alert enrichment fileSensitivity hiveResourceSensitivity ipzone mlmodel pcap pcapfiles streamMetadata streamdef t threatintel userprofile 23 row(s) in 0.3190 seconds nil TABLE ATLAS_ENTITY_AUDIT_EVENTS access_tracker alertDataSource alertExecutor alertStream alertStreamSchema alertdef alertdetail atlas_titan eagle_metric eaglehdfs_alert enrichment fileSensitivity hiveResourceSensitivity ipzone mlmodel pcap pcapfiles streamMetadata streamdef t threatintel userprofile 23 row(s) in 0.0170 seconds nil java exception ERROR Java::OrgApacheHadoopHbaseIpc::RemoteWithExtrasException: org.apache.hadoop.hbase.security.AccessDeniedException: org.apache.hadoop.security.AccessControlException: Permission denied. at org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.grant(RangerAuthorizationCoprocessor.java:1168) at org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos$AccessControlService$1.grant(AccessControlProtos.java:9933) at org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos$AccessControlService.callMethod(AccessControlProtos.java:10097) at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7717) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1897) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1879) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32299) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2127) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.security.AccessControlException: Permission denied. at org.apache.ranger.admin.client.RangerAdminRESTClient.grantAccess(RangerAdminRESTClient.java:168) at org.apache.ranger.plugin.service.RangerBasePlugin.grantAccess(RangerBasePlugin.java:308) at org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.grant(RangerAuthorizationCoprocessor.java:1161) ... 11 more ... View more

Re: Knox sso for ambari and ranger does not work

by Expert Contributor in Support Questions
‎02-23-2017 02:18 AM
‎02-23-2017 02:18 AM
yes,thank you,the knox host and ambari host should be the same domain suffix. i've solve this. ... View more

Re: Knox sso for ambari and ranger does not work

by Expert Contributor in Support Questions
‎02-22-2017 09:31 AM
‎02-22-2017 09:31 AM
the ambari and knox sso use the same user, and knox use ldap ... View more

Re: Knox sso for ambari and ranger does not work

by Expert Contributor in Support Questions
‎02-22-2017 09:29 AM
‎02-22-2017 09:29 AM
17/02/22 17:46:07 ||f67516cd-e553-43c8-9666-4dfd95b63a3c|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=http://bigdata6:8080/|unavailable|Request method: POST 17/02/22 17:46:07 ||f67516cd-e553-43c8-9666-4dfd95b63a3c|audit|KNOXSSO|venus|||authentication|uri|/gateway/knoxsso/api/v1/websso?originalUrl=http://bigdata6:8080/|success| 17/02/22 17:46:07 ||f67516cd-e553-43c8-9666-4dfd95b63a3c|audit|KNOXSSO|venus|||authentication|uri|/gateway/knoxsso/api/v1/websso?originalUrl=http://bigdata6:8080/|success|Groups: [] 17/02/22 17:46:07 ||f67516cd-e553-43c8-9666-4dfd95b63a3c|audit|KNOXSSO|venus|||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=http://bigdata6:8080/|success|Response status: 303 17/02/22 17:46:07 ||cc006ac5-1b98-4d20-bbdd-03a30f26fda4|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/redirecting.html?originalUrl=http://bigdata6:8080/|unavailable|Request method: GET 17/02/22 17:46:07 ||cc006ac5-1b98-4d20-bbdd-03a30f26fda4|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/redirecting.html?originalUrl=http://bigdata6:8080/|success|Response status: 200 17/02/22 17:46:07 ||2f023049-55b3-4bd9-879d-2430bde60f1f|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/bootstrap.min.css|unavailable|Request method: GET 17/02/22 17:46:07 ||2f023049-55b3-4bd9-879d-2430bde60f1f|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/bootstrap.min.css|success|Response status: 200 17/02/22 17:46:07 ||a0848c4b-637b-4699-8cec-efc85f425f6f|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/knox.css|unavailable|Request method: GET 17/02/22 17:46:07 ||a0848c4b-637b-4699-8cec-efc85f425f6f|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/knox.css|success|Response status: 200 17/02/22 17:46:07 ||cd5a3a24-5332-45c2-80b6-edbb8298cd07|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/images/loading.gif|unavailable|Request method: GET 17/02/22 17:46:07 ||cd5a3a24-5332-45c2-80b6-edbb8298cd07|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/images/loading.gif|success|Response status: 200 17/02/22 17:46:08 ||ded2eb86-5184-4c17-bfe2-ca557ae16fac|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=http%3A%2F%2Fbigdata6%3A8080%2F%23%2Flogin?redirected=true|unavailable|Request method: GET 17/02/22 17:46:08 ||ded2eb86-5184-4c17-bfe2-ca557ae16fac|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=http%3A%2F%2Fbigdata6%3A8080%2F%23%2Flogin?redirected=true|success|Response status: 401 17/02/22 17:46:08 ||6eb6e25a-4321-4c69-a7f5-aa7ea15ceb57|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/login.html?originalUrl=http%3A%2F%2Fbigdata6%3A8080%2F%23%2Flogin?redirected=true|unavailable|Request method: GET 17/02/22 17:46:08 ||6eb6e25a-4321-4c69-a7f5-aa7ea15ceb57|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/login.html?originalUrl=http%3A%2F%2Fbigdata6%3A8080%2F%23%2Flogin?redirected=true|success|Response status: 200 17/02/22 17:46:08 ||6e3bca36-1991-40bc-9587-fe35c3ecc61d|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/bootstrap.min.css|unavailable|Request method: GET 17/02/22 17:46:08 ||f355e30a-2159-42b9-8659-043dc3ef9496|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/knox.css|unavailable|Request method: GET 17/02/22 17:46:08 ||f355e30a-2159-42b9-8659-043dc3ef9496|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/knox.css|success|Response status: 200 17/02/22 17:46:08 ||6e3bca36-1991-40bc-9587-fe35c3ecc61d|audit|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/bootstrap.min.css|success|Response status: 200 this is log that visit one time ... View more

Re: Knox sso for ambari and ranger does not work

by Expert Contributor in Support Questions
‎02-22-2017 09:27 AM
‎02-22-2017 09:27 AM
<topology> <gateway> <provider> <role>webappsec</role> <name>WebAppSec</name> <enabled>true</enabled> <param><name>xframe.options.enabled</name><value>true</value></param> </provider> <provider> <role>authentication</role> <name>ShiroProvider</name> <enabled>true</enabled> <param> <name>sessionTimeout</name> <value>30</value> </param> <param> <name>redirectToUrl</name> <value>/gateway/knoxsso/knoxauth/login.html</value> </param> <param> <name>restrictedCookies</name> <value>rememberme,WWW-Authenticate</value> </param> <param> <name>main.ldapRealm</name> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value> </param> <param> <name>main.ldapContextFactory</name> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value> </param><param> <name>main.ldapRealm.contextFactory</name> <value>$ldapContextFactory</value> </param> <param> <name>main.ldapRealm.userDnTemplate</name> <value>uid={0},ou=people,dc=VENUS,dc=COM</value> </param> <param> <name>main.ldapRealm.contextFactory.url</name> <value>ldap://bigdata7:389</value> </param> <param> <name>main.ldapRealm.authenticationCachingEnabled</name> <value>false</value> </param> <param> <name>main.ldapRealm.contextFactory.authenticationMechanism</name> <value>simple</value> </param> <param> <name>urls./**</name> <value>authcBasic</value> </param> </provider> <provider> <role>identity-assertion</role> <name>Default</name> <enabled>true</enabled> </provider> </gateway> <application> <name>knoxauth</name> </application> <service> <role>KNOXSSO</role> <param> <name>knoxsso.cookie.secure.only</name> <value>false</value> </param> <param> <name>knoxsso.token.ttl</name> <value>30000</value> </param> <param> <name>knoxsso.redirect.whitelist.regex</name> <value>^https?:\/\/(bigdata[0-9]|localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*{replace15}lt;/value> </param></service> </topology> this is my knox sso topology， and my knox and ambari-server is not in the same machine. ... View more
Contact Me
Online
Offline
Last Visited
‎11-06-2015 02:24 AM
Community Statistics
Member Since ‎10-29-2015 10:08 PM
Last Visited ‎11-06-2015 02:24 AM
Posts 41
Kudos received 16