Having this issue in CM4.7, just wondering if its fixed in future release?
because my servers are mixed case, it failed to find the proper principal in the db:
Exception in secureMain
java.io.IOException: Login failure for hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM from keytab hdfs.keytab
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:825)
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:279)
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:243)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1726)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1751)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1904)
at org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:135)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:188)
Caused by: javax.security.auth.login.LoginException: Checksum failed
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:763)
at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:718)
at javax.security.auth.login.LoginContext.login(LoginContext.java:590)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:816)
... 11 more
As you can see here, regenerating the principals created new keytabs, for the principal with mixed case hostnames:
[root@1620-authentication krb5kdc]# kadmin.local
Authenticating as principal root/admin@BI2R.LEIDOS.COM with password.
kadmin.local: getprincs
HTTP/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM
HTTP/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM
HTTP/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM
HTTP/1620-cdh4u5.bi2r.leidos.com@BI2R.LEIDOS.COM
K/M@BI2R.LEIDOS.COM
bir/admin@BI2R.LEIDOS.COM
cloudera-scm/admin@BI2R.LEIDOS.COM
hdfs/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM
hdfs/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM
hdfs/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM
hdfs/1620-cdh4u5.bi2r.leidos.com@BI2R.LEIDOS.COM
hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM
hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM
hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM
...
The fix was to delete the principals I renamed last time, and then rename them:
kadmin.local: delprinc hdfs/1620-megatron.bi2r.leidos.com
Are you sure you want to delete the principal "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes
Principal "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM" deleted.
Make sure that you have removed this principal from all ACLs before reusing.
kadmin.local: delprinc hdfs/1620-node6.bi2r.leidos.com
Are you sure you want to delete the principal "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes
Principal "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM" deleted.
Make sure that you have removed this principal from all ACLs before reusing.
kadmin.local: delprinc hdfs/1620-scavenger.bi2r.leidos.com
Are you sure you want to delete the principal "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes
Principal "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM" deleted.
Make sure that you have removed this principal from all ACLs before reusing.
kadmin.local: renprinc hdfs/1620-Megatron.bi2r.leidos.com hdfs/1620-megatron.bi2r.leidos.com
Are you sure you want to rename the principal "hdfs/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM" to "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes
Principal "hdfs/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM" renamed to "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM".
Make sure that you have removed the old principal from all ACLs before reusing.
kadmin.local: renprinc hdfs/1620-Node6.bi2r.leidos.com hdfs/1620-node6.bi2r.leidos.com
Are you sure you want to rename the principal "hdfs/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM" to "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes
Principal "hdfs/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM" renamed to "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM".
Make sure that you have removed the old principal from all ACLs before reusing.
kadmin.local: renprinc hdfs/1620-Scavenger.bi2r.leidos.com hdfs/1620-scavenger.bi2r.leidos.com
Are you sure you want to rename the principal "hdfs/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM" to "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes
Principal "hdfs/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM" renamed to "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM".
Make sure that you have removed the old principal from all ACLs before reusing.
Interestingly enough, I only have this issue with hdfs and mapred roles… zookeeper, oozie and HTTP all respect the mixed case.
http://championofcyrodiil.blogspot.com
