How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principals

[#1]

I am on CDH 5.9.0 and using Cloudera Manager integrated with Active Directory to manage Kerberos ticket automatically. It is great until I am trying to enable Oozie HA via HAProxy.

How could I tell CM to generated HTTP keytab for oozie servers that contains HAProxy principal? I can do it manually. However, with CM Active Directory integration, I can't find a way to do so since I have no control of the keytab locations.

[#2]

If you look at the Oozie config page, and search for load balancer, is that configured correctly?

Did you set up HA for Oozie using the CM wizard?

https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_hag_oozie_ha.html

[#3]

I checked oozie.keytab which has http principals for both proxy and local host, so the key tab is generated fine. However, Web UI "Load Balancer" gives me HTTP Status 403 - GSSException: Failure unspecified at GSS-API level - Checksum failed. However, both individual OOzie Web UIs return fine.

I am using HAProxy. The proxy URL worked fine before enabling Kerberos. Is there any specific setting I should do in HAProxy?

[#4]

Yes. I enabled Oozie HA via CM.

[#5]

You can check in Administration --> Security

Click on "Kerberos Credentials"

You can search for the hostname you entered as the proxy to view the credentials that are stored in Cloudera Manager

Cloudera Manager will automatically merge the keytabs and lay down the proper keytab in the oozie process directory at the time it is started.  You can do a klist on the file.  You can see the latest process directory by running:

ls -lrt /var/run/cloudera-scm-agent/process |grep OOZIE

-Ben

[#6]

Ben,

CM did a good job on merging HTTP principals in oozie.keytab. However, my issue is the proxy. I got http 403 error on proxy UI, but not with two individual oozier server web UI.

[#7]

Can you share the full error?

What is the URL you used to try to access the UI?

[#8]

Web UI "Load Balancer" gives me HTTP Status 403 - GSSException: Failure unspecified at GSS-API level - Checksum failed. However, both individual OOzie Web UIs return fine.

[#9]

This sounds more like a server-side exception.  I recommend checking the Oozie logs for exceptions being thrown when attempting to access the UI via load balancer.  The exception should hopefully shed some light on what is happening.

You could shut down one Oozie instance to ensure you know which log to look at.

[#10 · Accepted solution]

Double-checked the KRB tickets, the principal for proxy is not using FQHN. I went back to check the LB configuration and sure it was using short name for the proxy host. Once I switched back, LB web UI comes back fine. Thanks.