Yarn RM is not getting started

[#1]

Hi,

I am using hdp 2.6 cluster with kerberos enabled. Yarn RM is not getting started and failing with the below error. I have kerberos enabled. Can any one please help me

ERROR client.ZooKeeperSaslClient (ZooKeeperSaslClient.java:createSaslToken(388)) - An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state. 2017-10-19 13:13:05,594 ERROR zookeeper.ClientCnxn (ClientCnxn.java:run(1059)) - SASL authentication with Zookeeper Quorum member failed: javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state. 2017-10-19 13:13:12,394 INFO recovery.ZKRMStateStore (ZKRMStateStore.java:runWithRetries(1227)) - Exception while executing a ZK operation. org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /rmstore

[#2]

@rmr1989

"Mechanism level: Server not found in Kerberos database" --> This is were the issue lies.

Please perform a forward and reverse DNS lookup of the server hostname. Your server has incorrect DNS entries. They are absolutely crucial for Kerberos. The proper place is your DNS server, in your case: domain controller. Figure out the IP address of your DNS server. Hope it Helps!

[#3]

Hi @Bala Vignesh N V

My DNS entries are looking good. The issue arises when I use custom service accounts for all the hadoop services. When I go with the default service accounts provided by Ambari, I am not facing this issue any more. I checked the keytabs, user principals for all the services including zookeeper. Everything looks good.