ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster

[#1]

Recently we had moved ambari-server to other host. cluster is kerberized.

metrics collector service dies after sometime with below error in ambari-metrics-collector.log

ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster

[#2 · Accepted solution]

this is resolved.

after checking /var/log/ambari-metrics-collector/hbase-ams-master-XXXX.log

found

Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user.

checked keytabs files under /etc/security/keytabs/ams-hbase.master.keytab

after klist to this keytab came to know that its has invalid entry for hosts.

klist -ekt ams-hbase.master.keytab

Keytab name: FILE:ams-hbase.master.keytab

KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (des3-cbc-sha1) 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (arcfour-hmac) 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (des3-cbc-sha1) 2 07/04/16 03:09:57 host/kdchost1.EXAMPLE.COM@EXAMPLE.COM (arcfour-hmac) 2 07/13/16 11:11:54 kmsusr@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 2 07/13/16 11:11:54 kmsusr@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 2 07/13/16 11:11:54 kmsusr@EXAMPLE.COM (des3-cbc-sha1) 2 07/13/16 11:11:54 kmsusr@EXAMPLE.COM (arcfour-hmac) 3 07/13/16 11:14:14 kmsusr@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 3 07/13/16 11:14:14 kmsusr@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 3 07/13/16 11:14:14 kmsusr@EXAMPLE.COM (des3-cbc-sha1) 3 07/13/16 11:14:14 kmsusr@EXAMPLE.COM (arcfour-hmac) 4 07/13/16 11:15:46 kmsusr@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 4 07/13/16 11:15:46 kmsusr@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 4 07/13/16 11:15:46 kmsusr@EXAMPLE.COM (des3-cbc-sha1) 4 07/13/16 11:15:46 kmsusr@EXAMPLE.COM (arcfour-hmac) 2 09/21/16 11:11:23 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 2 09/21/16 11:11:23 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 2 09/21/16 11:11:23 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (des3-cbc-sha1) 2 09/21/16 11:11:23 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (arcfour-hmac) 3 09/21/16 11:13:40 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 3 09/21/16 11:13:40 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 3 09/21/16 11:13:40 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (des3-cbc-sha1) 3 09/21/16 11:13:40 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (arcfour-hmac) 4 09/21/16 11:15:07 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 4 09/21/16 11:15:07 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (aes128-cts-hmac-sha1-96) 4 09/21/16 11:15:07 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (des3-cbc-sha1) 4 09/21/16 11:15:07 amshbase/edgenode.EXAMPLE.COM@EXAMPLE.COM (arcfour-hmac)

so deleted that keytab file and generated new keytab with valid entries and placed on the metrics-collector host under /etc/security/keytabs

restarted metrics-collector service the service started.