Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cloudera Manager CSD: want certain properties to be added/changed if cluster is kerberized

Cloudera Manager CSD: want certain properties to be added/changed if cluster is kerberized

I'm working on creating a CSD to integrate with Cloudera Manager. We have a configuation file (in hadoop xml format) that is being managed by the CSD, and there are certain properties in that file which need to be added (or changed), if and only if, the cluster is kerberized.

 

For example, if installing through Cloudera Manager on a non-kerberized cluster, the following properties might be set:

a.b.c = 1

d.e.f = 2

 

And if the cluster is kerberized, the properties might look like this:

a.b.c = 1

d.e.f = 1000

g.h.i = 7

 

 

The changes can be done manually, of course, but I'm hoping to be able to automate it. There is a way to do this in Ambari, and I'm looking for something similar. Is there a way to do this in the SDL file, or anywhere else?

 

I've gone through the documentation here: https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference, and the only kerberos references I see are with respect to adding principals.

3 REPLIES 3

Re: Cloudera Manager CSD: want certain properties to be added/changed if cluster is kerberized

Hi,

Ideally the underlying service would have a single toggle for Kerberos, rather than require the administrator / administrative tool know that several configs must be changed at the same time.

If that's not possible, then you can implement this in the bash script. There's many approaches to doing this, but in the simplest case, you can essentially:
1) Have a single parameter to control kerberos, let's say it's a boolean "kerberos.enabled".
2) In the SDL, have the kerberos section reference this parameter

"kerberos" : "${kerberos.enabled}",

3) Emit an environment variable

"roles": [
    {
      "name": "EXAMPLE_ROLE",
      // ...
      "startRunner": {
        // ...
        "environmentVariables": {
          "KERBEROS_ENABLED" : "${kerberos.enabled}",
          // ...

4) In your control script, check the value of $KERBEROS_ENABLED, and then add configs to the XML accordingly.

 

One nice way to accomplish the last step is to use additionalConfigs to create a placeholder value, then use perl to replace it. So your SDL would have:

"roles": [
    {
      "name": "EXAMPLE_ROLE",
      // ...
      "configWriter": {
        "generators": [
          {
            "filename": "example.xml",
            // ...
            "additionalConfigs" : [
              {
                "key" : "kerb.prop.a",
                "value" : "{{KERB_PROP_A}}"
              },
              {
                "key" : "kerb.prop.b",
                "value" : "{{KERB_PROP_B}}"
              },
              // ...

Then your control script would implement the conditional logic you need, then replace those variables:

if [[ ${KERBEROS_ENABLED} == "true" ]]; then
  KERB_PROP_A=1
  KERB_PROP_B=2
else
  KERB_PROP_A=100
  KERB_PROP_B=121
fi

perl -pi -e "s#\#{{KERB_PROP_A}}#${KERB_PROP_A}#" "$CONF_DIR/example.xml"
perl -pi -e "s#\#{{KERB_PROP_B}}#${KERB_PROP_B}#" "$CONF_DIR/example.xml"

Thanks,

Darren

Re: Cloudera Manager CSD: want certain properties to be added/changed if cluster is kerberized

Hi Darren,

 

I think that will functionally work, but when a user looks at properties in the Cloudera Manager UI, won't they see the old values? I think that will be confusing to a user.

 

Is there a way to update the values in the UI as well?

 

Thanks

Re: Cloudera Manager CSD: want certain properties to be added/changed if cluster is kerberized

Hi,

You're correct. CM Server displays the files that are sent to the agent before shell scripts are run, so they'll show the placeholder values if you used my suggested approach.

This is an unfortunate limitation. We don't have server-side support for this sort of advanced custom logic today. There's no way to update the UI as well.

I'll mention again that it's best if the underlying product doesn't require multiple configs to be changed when there's logically one change to be made, though I know that's not always something a CSD author can control.

Thanks,
Darren