Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas Metadata server not starting

Solved Go to solution
Highlighted

Atlas Metadata server not starting

Contributor

Hello Team,

After enabling Kerberos on HDP-2.6 cluster, Atlas metadata server is not starting. Getting following error:

java exception
ERROR Java::OrgApacheHadoopHbaseIpc::RemoteWithExtrasException: org.apache.hadoop.hbase.coprocessor.CoprocessorException: HTTP 503 Error: HTTP 503
    at org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.grant(RangerAuthorizationCoprocessor.java:1236)
    at org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos$AccessControlService$1.grant(AccessControlProtos.java:9933)
    at org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos$AccessControlService.callMethod(AccessControlProtos.java:10097)
    at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7857)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1999)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1981)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)

How to solve this? Please suggest.

Thanks,

Bhushan

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Atlas Metadata server not starting

Contributor

This issue occurred as we have used Load balancer while implementing Ranger HA. Breaking HA and keeping only one Ranger Admin resolved this issue for us.

3 REPLIES 3

Re: Atlas Metadata server not starting

Mentor

@Bhushan Kandalkar

This looks like an authorization issue in ranger, Atlas need the hbase DB for the backend ensure all hbase components are running, Check the in Ranger that Atlas has the correct permissions to 2 important tables Atlas_titan and ATLAS_ENTITY_AUDIT_EVENTS see attached screenshots

Log on the Zookeeper CLI and check the ACL's you should have a similar output

[zk: localhost:2181(CONNECTED) 0] getAcl  /hbase-unsecure/table/ATLAS_ENTITY_AUDIT_EVENTS
'world,'anyone
: cdrwa
[zk: localhost:2181(CONNECTED) 1] getAcl  /hbase-unsecure/table/atlas_titan
'world,'anyone
: cdrwa
[zk: localhost:2181(CONNECTED)2] getAcl  /hbase-unsecure/table/hbase:meta
'world,'anyone
: cdrwa
[zk: localhost:2181(CONNECTED) 3] getAcl  /hbase-unsecure/table/hbase:namespace
'world,'anyone
: cdrwa

Please revert


atlas-titan.pngatlas-entity-audit.png

Re: Atlas Metadata server not starting

Contributor

This issue occurred as we have used Load balancer while implementing Ranger HA. Breaking HA and keeping only one Ranger Admin resolved this issue for us.

Re: Atlas Metadata server not starting

Mentor

@Bhushan Kandalkar

Good it worked out but you shouldn't have omitted the information about the architecture ie Load balancer such info is critical in the analysis ....:-)

Happy hadooping