Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principals

SOLVED Go to solution

How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principals

Expert Contributor

I am on CDH 5.9.0 and using Cloudera Manager integrated with Active Directory to manage Kerberos ticket automatically. It is great until I am trying to enable Oozie HA via HAProxy.

 

How could I tell CM to generated HTTP keytab for oozie servers that contains HAProxy principal? I can do it manually. However, with CM Active Directory integration, I can't find a way to do so since I have no control of the keytab locations.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Expert Contributor

Double-checked the KRB tickets, the principal for proxy is not using FQHN. I went back to check the LB configuration and sure it was using short name for the proxy host. Once I switched back, LB web UI comes back fine. Thanks.

12 REPLIES 12

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

If you look at the Oozie config page, and search for load balancer, is that configured correctly?

Did you set up HA for Oozie using the CM wizard?

https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_hag_oozie_ha.html
Highlighted

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Expert Contributor

I checked oozie.keytab which has http principals for both proxy and local host, so the key tab is generated fine. However, Web UI "Load Balancer" gives me HTTP Status 403 - GSSException: Failure unspecified at GSS-API level - Checksum failed. However, both individual OOzie Web UIs return fine.

 

I am using HAProxy. The proxy URL worked fine before enabling Kerberos. Is there any specific setting I should do in HAProxy?

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Expert Contributor

Yes. I enabled Oozie HA via CM.

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Super Guru

You can check in Administration --> Security

Click on "Kerberos Credentials"

 

You can search for the hostname you entered as the proxy to view the credentials that are stored in Cloudera Manager

 

Cloudera Manager will automatically merge the keytabs and lay down the proper keytab in the oozie process directory at the time it is started.  You can do a klist on the file.  You can see the latest process directory by running:

 

ls -lrt /var/run/cloudera-scm-agent/process |grep OOZIE

 

 

-Ben

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Expert Contributor

Ben,

CM did a good job on merging HTTP principals in oozie.keytab. However, my issue is the proxy. I got http 403 error on proxy UI, but not with two individual oozier server web UI.

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Super Guru

Can you share the full error?

What is the URL you used to try to access the UI?

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Expert Contributor

Web UI "Load Balancer" gives me HTTP Status 403 - GSSException: Failure unspecified at GSS-API level - Checksum failed. However, both individual OOzie Web UIs return fine.

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Super Guru

This sounds more like a server-side exception.  I recommend checking the Oozie logs for exceptions being thrown when attempting to access the UI via load balancer.  The exception should hopefully shed some light on what is happening.

You could shut down one Oozie instance to ensure you know which log to look at.

 

 

Re: How to merge oozie HA HTTP keberos principals with Cloudera Manager handling Kerboeros principal

Expert Contributor

Double-checked the KRB tickets, the principal for proxy is not using FQHN. I went back to check the LB configuration and sure it was using short name for the proxy host. Once I switched back, LB web UI comes back fine. Thanks.