Issue with mixed case server names

charlescva — Fri, 16 Sep 2022 08:54:17 GMT

Having this issue in CM4.7, just wondering if its fixed in future release?

because my servers are mixed case, it failed to find the proper principal in the db:

Exception in secureMain

java.io.IOException: Login failure for hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM from keytab hdfs.keytab

at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:825)

at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:279)

at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:243)

at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1726)

at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1751)

at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1904)

at org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:135)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:188)

Caused by: javax.security.auth.login.LoginException: Checksum failed

at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:763)

at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)

at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)

at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721)

at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:718)

at javax.security.auth.login.LoginContext.login(LoginContext.java:590)

at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:816)

... 11 more

As you can see here, regenerating the principals created new keytabs, for the principal with mixed case hostnames:

[root@1620-authentication krb5kdc]# kadmin.local

Authenticating as principal root/admin@BI2R.LEIDOS.COM with password.

kadmin.local: getprincs

HTTP/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM

HTTP/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM

HTTP/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM

HTTP/1620-cdh4u5.bi2r.leidos.com@BI2R.LEIDOS.COM

K/M@BI2R.LEIDOS.COM

bir/admin@BI2R.LEIDOS.COM

cloudera-scm/admin@BI2R.LEIDOS.COM

hdfs/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM

hdfs/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM

hdfs/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM

hdfs/1620-cdh4u5.bi2r.leidos.com@BI2R.LEIDOS.COM

hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM

hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM

hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM

...

The fix was to delete the principals I renamed last time, and then rename them:

kadmin.local: delprinc hdfs/1620-megatron.bi2r.leidos.com

Are you sure you want to delete the principal "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes

Principal "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM" deleted.

Make sure that you have removed this principal from all ACLs before reusing.

kadmin.local: delprinc hdfs/1620-node6.bi2r.leidos.com

Are you sure you want to delete the principal "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes

Principal "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM" deleted.

Make sure that you have removed this principal from all ACLs before reusing.

kadmin.local: delprinc hdfs/1620-scavenger.bi2r.leidos.com

Are you sure you want to delete the principal "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes

Principal "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM" deleted.

Make sure that you have removed this principal from all ACLs before reusing.

kadmin.local: renprinc hdfs/1620-Megatron.bi2r.leidos.com hdfs/1620-megatron.bi2r.leidos.com

Are you sure you want to rename the principal "hdfs/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM" to "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes

Principal "hdfs/1620-Megatron.bi2r.leidos.com@BI2R.LEIDOS.COM" renamed to "hdfs/1620-megatron.bi2r.leidos.com@BI2R.LEIDOS.COM".

Make sure that you have removed the old principal from all ACLs before reusing.

kadmin.local: renprinc hdfs/1620-Node6.bi2r.leidos.com hdfs/1620-node6.bi2r.leidos.com

Are you sure you want to rename the principal "hdfs/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM" to "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes

Principal "hdfs/1620-Node6.bi2r.leidos.com@BI2R.LEIDOS.COM" renamed to "hdfs/1620-node6.bi2r.leidos.com@BI2R.LEIDOS.COM".

Make sure that you have removed the old principal from all ACLs before reusing.

kadmin.local: renprinc hdfs/1620-Scavenger.bi2r.leidos.com hdfs/1620-scavenger.bi2r.leidos.com

Are you sure you want to rename the principal "hdfs/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM" to "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM"? (yes/no): yes

Principal "hdfs/1620-Scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM" renamed to "hdfs/1620-scavenger.bi2r.leidos.com@BI2R.LEIDOS.COM".

Make sure that you have removed the old principal from all ACLs before reusing.

Interestingly enough, I only have this issue with hdfs and mapred roles… zookeeper, oozie and HTTP all respect the mixed case.

Re: Issue with mixed case server names

Darren — Tue, 18 Feb 2014 21:56:01 GMT

Unfortunately, host names with capital letters will always hit this problem. CM respects the original host name capitalization, but hadoop converts it all to lowercase.

You'll have to pick lowercase host names. You might also be able to change your agent.ini on each host to override the hostname to be the lowercase name, haven't tried that though.

Thanks,

Darren

question Re: Issue with mixed case server names in Archives of Support Questions (Read Only)

Issue with mixed case server names

Re: Issue with mixed case server names

Re: Issue with mixed case server names