Remark regarding Java 8 Kerberos configuration compability

[#1]

Following Cloudera guide on implementing Java 8 (https://www.cloudera.com/documentation/enterprise/upgrade/topics/ug_jdk8.html) I found the following issue that I think may be of interest to others.

Because of this Java bug: JDK-8029994 : Support "include" and "includedir" in krb5.conf (https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8029994) Java 8 does not support includedir/include directives in krb5.conf.

Standard CentOS/Redhat Kerberos installation may have includedir /etc/krb5.conf.d/ in /etc/krb5.conf. Using sssd may add includedir /var/lib/sss/pubconf/krb5.include.d/ as well. Both lines in the configuration file will cause an error.

Java 7 was based on Sun/Oracle implementation, so I think the problem is essentially caused by lack of port of said feature.

Java 7 code with feature: http://cr.openjdk.java.net/~weijun/8031111/webrev.00/src/java.security.jgss/share/classes/sun/securi...

Java 8 code without feature: http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/f0b93fbd8cf8/src/share/classes/sun/security/krb5/Con...

Workaround is to include the content of the included files directly. Downside is that any changes that patches will make to eg. sssd krb5.conf will no be picked up automatically.

If this is not implemented prior to switching Cloudera Manager services and Hadoop services to Java 8 the following exception wiil occur when trying to start various serices eg. Activity Monitor, Zookeeper and HDFS.

Caused by: KrbException: Illegal config content:includedir /etc/krb5.conf.d/
at sun.security.krb5.Config.parseStanzaTable(Config.java:634)
at sun.security.krb5.Config.<init>(Config.java:197)
at sun.security.krb5.Config.getInstance(Config.java:98

This issue is also present in Oracle' Java - not just OpenJDK.

The above experiences are based on  Java 8u202 (Oracle' edition) and CM/CDH 5.16.1

[#2]

Hello @henrikringcsc ,

Thank you for bringing up this topic and doing so much research!

The bug you mention was fixed some time ago, so what you are seeing now is different than the older bugs.

The error you see is:

Caused by: KrbException: Illegal config content:includedir /etc/krb5.conf.d/

The error from the Java bug is:

javax.security.auth.login.LoginException: KrbException: Config file must starts with a section

Based on the exception, your version, and the current Kerberos Configuration file parsing "Config" class, we can guess that your krb5.conf had an "includedir" inside a section of krb5.conf which is, indeed, not legal for MIT or Java Kerberos implementations.  To put the theory to the test, I tested with Oracle JDK 1.8u162 and OpenJDK 1.8u201.  When I had the following in my krb5.conf file, my HTTPFS server started fine:

# Other applications require this directory to perform krb5 configuration.
includedir /etc/krb5.conf.d/
[libdefaults]

If, however, I added the "includedir" line after the libdefaults section title, I reproduced your exception.  I did this:

# Other applications require this directory to perform krb5 configuration.
includedir /etc/krb5.conf.d/
[libdefaults]
includedir /etc/krb5.conf.d/

The exception when starting HTTPFS server (snipped):

Caused by: KrbException: Illegal config content:includedir /etc/krb5.conf.d/
        at sun.security.krb5.Config.parseStanzaTable(Config.java:634)
        at sun.security.krb5.Config.<init>(Config.java:197)
        at sun.security.krb5.Config.getInstance(Config.java:98)

Looking at the latest code here, we can then see that parseStanzaTable comes after the krb5.conf file has been read into memory.  In https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8029994 we see that the bug was in the loading of the configuration file, not in the parsing of the sections as we see in your exception.

http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/4e2fc4ce3a1a/src/share/classes/sun/security/krb5/Con...

CONCLUSION:

If you wish to add "includedir" lines, they are fine, but they need to come before the first section title in your krb5.conf file in order for it to be valid.

If that doesn't work out, let us know and include your krb5.conf so we can understand how it is formatted.

Ben

[#3]

Hi Ben ( @bgooley )

Thank you taking the time to investigate and comment.
Your help is very much appreciated.

I am using CM to manage krb5.conf ("Manage krb5.conf through Cloudera Manager" is checked).

If what you are saying about the placement of includedir in the krb5.conf file is correct - that it must come before the first section - I am having problems using this feature.

There are 3 configuration CM parameter that would enable me to add an includedir to krb5.conf:

• Advanced Configuration Snippet (Safety Valve) for [libdefaults] section of krb5.conf
• Advanced Configuration Snippet (Safety Valve) for the Default Realm in krb5.conf
• Advanced Configuration Snippet (Safety Valve) for remaining krb5.conf

Neither of them are injected into krb5.conf before the first section [libdefault].

So I cannot use includedir with CM managing krb5.conf

Second. I am reading the MIT documentation (https://web.mit.edu/kerberos/krb5-1.15/doc/admin/conf_files/krb5_conf.html😞

The krb5.conf file can include other files using either of the following directives at the beginning of a line:
include FILENAME
includedir DIRNAME
FILENAME or DIRNAME should be an absolute path ...

I cannot se any requirement that includir should be at the beginning of the file; only at the beginning of the line.

Having an includedir at the end of the krb5.conf file does indeed work with the Linux OS (CentOS 7.5).
It is only Java 8+ that is having an issue.

So I will still claim that somewhere ends do not meet up.

Either CM has insuficcient features to deal with scenarios where includedir is used (which is default if sssd is used), or Java 8 is missing af feature that was present in Java 7.

BUT as I described in my initial post a workaround is available (and fairly benign), one just needs to be aware of the issue if a Java 8 migration is in the making.

Once again thank you for your insights.

Best regards,
Henrik Ring