Support Questions

cjervis · ‎11-06-2019

Hi.

I'm kerberizing an EDH 6.1.1 on SLES on Google GCE.

Kerberos is up and running.

I've created a cloudera-scm/admin@REALM and gained its ticket with kinit, so I suppose everything is fine.

I try to enable Kerberos using the CM wizard.

When I reach the Setup KDC Account I set the cloudera-scm@REALM principal and the Continue button is enabled.

I try to click on it to proceed but nothing happens.

I took a look at krb5 log files and to cloudera-scm-server log files but nothing seems to be logged.

I've doublechecked NTP, Reverse DNS and krb5.conf file but found nothing useful.

Any help ?

Thanks

Stefano

svasi · ‎01-24-2020

Hi @Shelton

Finally I've solved the problem and hope my experience could help someone else who is banging his/her head on this scenario.

At time of writing I was accessing the GCP cluster via public IP addresses.

I had a /etc/hosts file resolving hostnames to their public IP

Then customer's NOC granted me access from internal network over VPN and next attempt went fine.

Still can't understand why, maybe some networking related config issue, but being on the internal side of the network solved my problem

Simple as that!

Thanks for your precious support, and patience

Stefano

View solution in original post

Shelton · ‎11-07-2019

@svasi

Any feedback on this issue?

svasi · ‎11-07-2019

I've solved the UDP port 88 problem it was a dead kadmid process.

I've killed it and restarted Kerberos services.

Now both krb5kdc and kadmind are up without issues.

Here their log files

krb5kd5.log

Nov 08 07:35:03 master-1 krb5kdc[22649](info): setting up network...
Nov 08 07:35:03 master-1 krb5kdc[22649](info): setting up network...
Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 08 07:35:03 master-1 krb5kdc[22649](info): set up 2 sockets
Nov 08 07:35:03 master-1 krb5kdc[22649](info): set up 2 sockets
Nov 08 07:35:03 master-1 krb5kdc[22650](info): commencing operation
Nov 08 07:35:03 master-1 krb5kdc[22650](info): commencing operation

kadmind.log

Nov 08 07:35:26 master-1 kadmind[22689](info): setting up network...
Nov 08 07:35:26 master-1 kadmind[22689](info): setting up network...
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo)
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo)
kadmind: setsockopt(10,IPV6_V6ONLY,1) worked
kadmind: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 10: udp ::.464 (pktinfo)
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 10: udp ::.464 (pktinfo)
kadmind: setsockopt(11,IPV6_V6ONLY,1) worked
kadmind: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 12: tcp 0.0.0.0.464
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 12: tcp 0.0.0.0.464
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 11: tcp ::.464
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 11: tcp ::.464
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 13: rpc 0.0.0.0.749
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 13: rpc 0.0.0.0.749
kadmind: setsockopt(14,IPV6_V6ONLY,1) worked
kadmind: setsockopt(14,IPV6_V6ONLY,1) worked
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 14: rpc ::.749
Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 14: rpc ::.749
Nov 08 07:35:26 master-1 kadmind[22689](info): set up 6 sockets
Nov 08 07:35:26 master-1 kadmind[22689](info): set up 6 sockets
Nov 08 07:35:26 master-1 kadmind[22690](info): Seeding random number generator
Nov 08 07:35:26 master-1 kadmind[22690](info): Seeding random number generator
Nov 08 07:35:26 master-1 kadmind[22690](info): starting
Nov 08 07:35:26 master-1 kadmind[22690](info): starting

Socket status

master-1:~ #  netstat -tupln|grep 749
tcp        0      0 0.0.0.0:749             0.0.0.0:*               LISTEN      22452/kadmind
tcp        0      0 :::749                  :::*                    LISTEN      22452/kadmind
master-1:~ # netstat -tupln|grep 88
tcp        0      0 10.243.1.196:8088       0.0.0.0:*               LISTEN      2050/java
tcp        0      0 0.0.0.0:18088           0.0.0.0:*               LISTEN      2048/java
tcp        0      0 10.243.1.196:19888      0.0.0.0:*               LISTEN      2461/java
udp        0      0 0.0.0.0:88              0.0.0.0:*                           22439/krb5kdc
udp        0      0 :::88                   :::*                                22439/krb5kdc

Anyway I still can't proceed with Cloudera kerberization because when I push (several times) the CONTINUE button in the right lower corner, which is enabled, nothing happens and the wizard remains on the Setup KDC Account page.

Here's a screenshot from the wizard where I try to click, even several times, but nothing happens.

I've tried with Mozilla, Edge, Chrome and IE browsers with no luck.

If I change the principal realm in the wizard, cloudera manager raises a warning (see below picture) and the CONTINUE button is disabled.

So it doesn't seem to be a Kerberos related issue.

Cloudera manager seems aware of Kerberos configuration.

From shell I can work on principals, adding or deleting, obtain tickets with kinit and destroy them with kdestroy.

I've set Cloudera Manager logs at DEBUG level but nothing is traced, neither in the above kerberos log files, when I click the above button.

Really can't figure out where the problem is.

Any idea?

svasi · ‎11-08-2019

I'm reading again Kerberos docs and I had doubt on my krb5.conf domain realm section.

My network domain is dev.edl.gcp.domain.it so I've added its realm translation but still not working in cloudera manager.

[domain_realm]
        .edhdev.com = EDHDEV.COM
        edhdev.com = EDHDEV.COM
        .dev.edl.gcp.domain.it = EDHDEV.COM
        dev.edl.gcp.domain.it = EDHDEV.COM

So I've removed it.

svasi · ‎11-08-2019

Just tried to set up a different cluster, also on Google Cloud Platform, with a CDH 5.15 to try a new setup on another version.

Same configuration, same behaviour.

Another setup local to my datacenter has no problem.

Does anybody met same problem on GCP?

Shelton · ‎11-08-2019

@svasi

WOW we have been going around for some time now, you realize its always very important to give a comprehensive description of your environment this helps quickly zero on the problem.

I have never deployed on GCP so I can't be of much help on the platform side.

Below is a document you should have gone through to give you pointers and avoid the frustration 🙂

https://docs.cloudera.com/documentation/director/latest/topics/director_get_started_gcp_install_cm_c...

Please read this document and revert

svasi · ‎11-11-2019

Thanks for your indications.

I've deployed a 5.15, on RedHat, using Altus Director on GCP.

Same behaviour and stuck in the same place.

Shelton · ‎11-11-2019

@svasi

Sorry that nothing is working out for you, my guess is the GCP platform, Have you seen this link using bdutil?

Installing HDP on GCP

I am wondering what documentation your are following can you share the link I have some free credit I could try that out this weekend

svasi · ‎11-18-2019

Hi.

Sorry for my late reply.

I've never met that link and I'll study it.

I'm following standard CDH 6.1.1 installation found here: https://docs.cloudera.com/documentation/enterprise/6/6.1/topics/cm_sg_authentication.html#xd_583c10b...

Shelton · ‎11-18-2019

@svasi

read through the documentation in that link and let me know !

svasi · ‎11-24-2019

Back again.

Followed, step by step, the document but the problem is still here.

Cloudera Community

Support Questions

Enable Kerberos - Setup KDC Account - Continue enabled but not working