[Kerberos] How to setup Cloudera Manager in order to use a different krb5.conf file/location?

[#1]

Hi.

I've been able to configure our cluster with a local kerberos realm using the usual /etc/krb5.conf file.

The thing is... we need to change the path that the system will use. 

Basically the customer is using the path /etc/krb5.conf with something else, but we need to setup Cloudera Manager (and the rest of the big data services) to use a different krb5.conf location.

Anyone know how to perform this change in Cloudera Manager (cluster wide)? To use a different krb5.conf file, other than the usual /etc/krb5.conf?

Thanks... :'(

[#2 · Accepted solution]

@JoaoBarreto,

Currently there is no way to automate this in Cloudera Manager, but it is possible via manual configuration for each service, the agents, and Cloudera Manager (as well as the shell if you are using hadoop commands at the command line).

 I would like to do some extensive testing at some point, but, for now, you can use the following:

Background:

Java will use the following configuration if set for that JVM:

-Djava.security.krb5.conf=/custom/path/to/krb5.conf

If java.security.krb5.conf is not set, then java will look in the following locations:

- /path_to_jdk/jre/lib/security/krb5.conf

- /etc/krb5.conf

MIT Kerberos-based servers can be configured with the following environment variable:

KRB5_CONFIG=/custom/path/to/krb5.conf

With the above rules in mind, these general steps can be followed:

(1)

Place your custom krb5.conf in the "jre/lib/security" subdirectory of your JDK's directory.

Make sure it is read for all in terms of file permissions (so that all service users can read from it)

Any client or server that uses that JDK will then automatically read from your customer krb5.conf rather than /etc.krb5.conf

* this includes Cloudera Manager

NOTE:  The draw-back of doing it this way is that if you upgrade Java, you will need to remember to put your krb5.conf in place. 

NOTE2: If you choose to use -Djava.security.krb5.conf instead, that will require configuring it for all servers and clients in safety valves, files, etc.  The plus of this config, though, is that you do not have to remember to put your krb5.conf in place during upgrades of JDK.

(2)

For all agents in your cluster, add this to /etc/default/cloudera-scm-agent:

export KRB5_CONFIG=/custom/path/to/krb5.conf

(3)

Add the following to Hue Service Environment Advanced Configuration Snippet (Safety Valve)

KRB5_CONFIG=/custom/path/to/krb5.conf

(4)

Add the following to Impala Service Environment Advanced Configuration Snippet (Safety Valve)

KRB5_CONFIG=/custom/path/to/krb5.conf

You may also need to add to Impala Daemon Environment Advanced Configuration Snippet (Safety Valve):

JAVA_TOOL_OPTIONS="-Djava.security.krb5.conf=/opt/krb5.conf

(5)

Restart EVERYTHING (cluster, management service, agents, Cloudera Manager)

That should give you a good start.

[#3]

Dude, with this reply you should really get a raise!

Thanks! We will test this solution! Thank you so much!

____________________________________________________

After some tests the solution is correct! It worked 🙂 

[#4]

Just a few changes I've detected.

In the agent configuration don't use EXPORT. 

In /etc/default/cloudera-scm-agent just add:

KRB5_CONFIG=/path/krb5.conf

Also, you will need to hammer Kerberos Server and Kadmin files.

KDC -> /etc/sysconfig/krb5kdc

Add: KRB5_CONFIG=/path/krb5.conf

And also:

Kadmin -> /etc/sysconfig/kadmin

Add: KRB5_CONFIG=/path/krb5.conf

This will allow you to start both services after you create the databases (kdb5_util create -s). If you don't do this, kerberos will still read /etc/krb5.conf and weird stuff will appear. 

LET'S GO PEOPLE!!! 😄 Hammer on!