question Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working in Support Questions

Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Fri, 08 Nov 2019 14:42:58 GMT

Hi.

I'm kerberizing an EDH 6.1.1 on SLES on Google GCE.

Kerberos is up and running.

I've created a cloudera-scm/admin@REALM and gained its ticket with kinit, so I suppose everything is fine.

I try to enable Kerberos using the CM wizard.

When I reach the Setup KDC Account I set the cloudera-scm@REALM principal and the Continue button is enabled.

I try to click on it to proceed but nothing happens.

I took a look at krb5 log files and to cloudera-scm-server log files but nothing seems to be logged.

I've doublechecked NTP, Reverse DNS and krb5.conf file but found nothing useful.

Any help ?

Thanks

Stefano

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

Shelton — Wed, 06 Nov 2019 17:39:47 GMT

@svasi

Can you explain one cannot guess with the information you have given.

HDP or CDH?

Steps executed

Krb5.conf

Kdc.conf

kadm5.acl

Error log or screenshot.

With the above the problem could be easier to analyze

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Wed, 06 Nov 2019 18:08:36 GMT

Thanks @Shelton

Is a CDH 6.1.1 on SUSE Enterprise 12.4 running on Google GCP compute engines.

I've installed, with zypper, those packages on the CM server:

zypper install krb5-server krb5-client krb5 openldap2-client

This is the /etc/krb5.conf file

[libdefaults] # "dns_canonicalize_hostname" and "rdns" are better set to false for improved security. # If set to true, the canonicalization mechanism performed by Kerberos client may # allow service impersonification, the consequence is similar to conducting TLS certificate # verification without checking host name. # If left unspecified, the two parameters will have default value true, which is less secure. dns_canonicalize_hostname = false rdns = false default_realm = EDHDEV.COM default_ccache_name = FILE:/tmp/krb5cc_%{uid} [realms] EDHDEV.COM = { kdc = master-1.dev.edl.gcp.domain.it admin_server = master-1.dev.edl.gcp.domain.it } [logging] kdc = FILE:/var/log/krb5/krb5kdc.log admin_server = FILE:/var/log/krb5/kadmind.log default = SYSLOG:NOTICE:DAEMON

This is the kdc.conf file

[kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88 [realms] EDHDEV.COM = { database_name = /var/lib/kerberos/krb5kdc/principal admin_keytab = FILE:/var/lib/kerberos/krb5kdc/kadm5.keytab acl_file = /var/lib/kerberos/krb5kdc/kadm5.acl dict_file = /var/lib/kerberos/krb5kdc/kadm5.dict key_stash_file = /var/lib/kerberos/krb5kdc/.k5.EDHDEV.COM kdc_ports = 88 max_life = 1d max_renewable_life = 7d } [logging] kdc = FILE:/var/log/krb5/krb5kdc.log admin_server = FILE:/var/log/krb5/kadmind.log

This is the kadm5.acl file:

############################################################################### #Kerberos_principal permissions [target_principal] [restrictions] ############################################################################### # */admin@EDHDEV.COM *

I've created, using kadmin.local, the cloudera-scm/admin@EDHDEV.COM

master-1:~ # kadmin.local Authenticating as principal cloudera-scm/admin@EDHDEV.COM with password. kadmin.local: listprincs K/M@EDHDEV.COM cloudera-scm/admin@EDHDEV.COM kadmin/admin@EDHDEV.COM kadmin/changepw@EDHDEV.COM kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM krbtgt/EDHDEV.COM@EDHDEV.COM

I can authenticate against kerberos with kinit and gain the ticket (It takes arount 8 seconds to get the ticket):

master-1:~ # kinit cloudera-scm/admin@EDHDEV.COM Password for cloudera-scm/admin@EDHDEV.COM: master-1:~ # klist -e Ticket cache: FILE:/tmp/krb5cc_0 Default principal: cloudera-scm/admin@EDHDEV.COM Valid starting Expires Service principal 11/06/19 17:58:08 11/07/19 17:58:08 krbtgt/EDHDEV.COM@EDHDEV.COM Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

I run the wizard in CM to enable Kerberos on my cluster.

I set KDC host and Encryption type according to previous output

Here I configure the principal but nothing happens when I click on Continue:

Kerberos log files:

==> /var/log/krb5/kadmind.log <== Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator Nov 06 12:37:51 master-1 kadmind[1109](info): starting Nov 06 12:37:51 master-1 kadmind[1109](info): starting Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one. Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one. Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one. Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one. ==> /var/log/krb5/krb5kdc.log <== krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation

cloudera-scm-server.log when I change from manage krb5.conf page to Setup KDC Account one and, then, select Continue button

2019-11-06 17:49:40,620 INFO scm-web-154:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/v15/batch 2019-11-06 17:49:40,625 INFO scm-web-154:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/v15/batch, Status:200 2019-11-06 17:49:40,885 INFO scm-web-380:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/v15/batch 2019-11-06 17:49:40,889 INFO scm-web-380:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/v15/batch, Status:200

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

Shelton — Wed, 06 Nov 2019 20:43:23 GMT

@svasi

I saw a couple of potential issues and I would like you to make a few changes.

Recreate an admin for Kerberos

Using the below syntax

# kadmin.local -q "addprinc admin/admin"

----------start of output---------
Authenticating as principal root/admin@EDHDEV.COM with a password.
WARNING: no policy specified for admin/admin@EDHDEV.COM; defaulting to no policy
Enter the password for principal "admin/admin@EDHDEV.COM": welcome1
Re-enter password for principal "admin/admin@EDHDEV.COM": welcome1
Principal "admin/admin@EDHDEV.COM" created.
----------end of output---------

Validate the principal works

# kadmin -p admin/admin@EDHDEV.COM

It will ask for the password once successfully authenticated then type q to quit

Can you leave the default encryption types? and you should select Manage krb5.conf through Cloudera Manager

Don't change the kadm5.acl file it's okay the way it is !!

I saw something missing in your krb5.conf between [realms] and [logging] the domain_realm add that to all the hosts if it's a multinode

[realms]
EDHDEV.COM = {
kdc = master-1.dev.edl.gcp.domain.it
admin_server = master-1.dev.edl.gcp.domain.it
}

[domain_realm]
.edhdev.com = EDHDEV.COM
edhdev.com = EDHDEV.COM

[logging]
kdc = FILE:/var/log/krb5/krb5kdc.log
admin_server = FILE:/var/log/krb5/kadmind.log
default = SYSLOG:NOTICE:DAEMON

Go to the CM and launch the kerberization using this below the principal

admin/admin@EDHDEV.COM

Please make the above changes restart the kdc and retry kerberization it should go smoothly

Happy hadooping

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Thu, 07 Nov 2019 10:21:06 GMT

Thanks @Shelton

Just followed your post but I still can't continue over Setup KDC Account step.

##### CREATED ADMIN PRINCIPAL master-1:~ # kadmin.local -q "addprinc admin/admin" Authenticating as principal cloudera-scm/admin@EDHDEV.COM with password. WARNING: no policy specified for admin/admin@EDHDEV.COM; defaulting to no policy Enter password for principal "admin/admin@EDHDEV.COM": Re-enter password for principal "admin/admin@EDHDEV.COM": Principal "admin/admin@EDHDEV.COM" created. ......................................................................... ##### VALIDATED ADMIN PRINCIPAL master-1:~ # kadmin -p admin/admin@EDHDEV.COM Authenticating as principal admin/admin@EDHDEV.COM with password. Password for admin/admin@EDHDEV.COM: kadmin: q

Continue button is enabled but pressing it does nothing and procedure remains on the same page.

Nothing is written in krb5 or cloudera manager log files.

I've raised cloudera manager's log4j level to DEBUG but still nothing is traced when I press Continue button

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

Shelton — Thu, 07 Nov 2019 12:02:57 GMT

@svasi

Can you share the files mentioned in my previous post? Redact site specific sensitive information and I will have a look at it.

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Thu, 07 Nov 2019 12:39:43 GMT

krb5.conf

[libdefaults] # "dns_canonicalize_hostname" and "rdns" are better set to false for improved security. # If set to true, the canonicalization mechanism performed by Kerberos client may # allow service impersonification, the consequence is similar to conducting TLS certificate # verification without checking host name. # If left unspecified, the two parameters will have default value true, which is less secure. dns_canonicalize_hostname = false rdns = false default_realm = EDHDEV.COM default_ccache_name = FILE:/tmp/krb5cc_%{uid} [realms] EDHDEV.COM = { kdc = master-1.dev.edl.gcp.domain.it admin_server = master-1.dev.edl.gcp.domain.it } [domain_realm] .edhdev.com = EDHDEV.COM edhdev.com = EDHDEV.COM [logging] kdc = FILE:/var/log/krb5/krb5kdc.log admin_server = FILE:/var/log/krb5/kadmind.log default = SYSLOG:NOTICE:DAEMON

kdc.conf

kadm5.acl

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

Shelton — Thu, 07 Nov 2019 12:57:05 GMT

@svasi

Sorry also these 2 logs the files attached look okay,

/var/log/krb5/krb5kdc.log
/var/log/krb5/kadmind.log

Did you create the re-create the principal? Share the output of

# kadmin.local

listprincs

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Thu, 07 Nov 2019 14:39:47 GMT

kadmin.local

listprincs

master-1:/var/log/krb5 # kadmin.local Authenticating as principal admin/admin@EDHDEV.COM with password. kadmin.local: listprincs K/M@EDHDEV.COM admin/admin@EDHDEV.COM kadmin/admin@EDHDEV.COM kadmin/changepw@EDHDEV.COM kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM krbtgt/EDHDEV.COM@EDHDEV.COM

kadmin.log

Nov 06 09:39:36 master-1 kdb5_util[30635](info): No dictionary file specified, continuing without one. Nov 06 09:39:36 master-1 kdb5_util[30635](info): No dictionary file specified, continuing without one. Nov 06 09:42:18 master-1 kadmin.local[31087](info): No dictionary file specified, continuing without one. Nov 06 09:42:18 master-1 kadmin.local[31087](info): No dictionary file specified, continuing without one. Nov 06 12:37:51 master-1 kadmind[1083](info): setting up network... Nov 06 12:37:51 master-1 kadmind[1083](info): setting up network... Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo) Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo) kadmind: setsockopt(10,IPV6_V6ONLY,1) worked kadmind: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 10: udp ::.464 (pktinfo) Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 10: udp ::.464 (pktinfo) kadmind: setsockopt(11,IPV6_V6ONLY,1) worked kadmind: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 12: tcp 0.0.0.0.464 Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 12: tcp 0.0.0.0.464 Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 11: tcp ::.464 Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 11: tcp ::.464 Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 13: rpc 0.0.0.0.749 Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 13: rpc 0.0.0.0.749 kadmind: setsockopt(14,IPV6_V6ONLY,1) worked kadmind: setsockopt(14,IPV6_V6ONLY,1) worked Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 14: rpc ::.749 Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 14: rpc ::.749 Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator Nov 06 12:37:51 master-1 kadmind[1109](info): starting Nov 06 12:37:51 master-1 kadmind[1109](info): starting Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one. Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one. Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one. Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one. Nov 07 09:10:10 master-1 kadmind[1109](debug): Got signal to request exit Nov 07 09:10:10 master-1 kadmind[1109](debug): Got signal to request exit Nov 07 09:10:10 master-1 kadmind[1109](info): finished, exiting Nov 07 09:10:10 master-1 kadmind[1109](info): finished, exiting Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 14 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 14 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 13 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 13 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 11 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 11 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 12 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 12 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 10 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 10 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 9 Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 9 Nov 07 09:11:13 master-1 kadmind[1091](info): setting up network... Nov 07 09:11:13 master-1 kadmind[1091](info): setting up network... Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo) Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo) kadmind: setsockopt(10,IPV6_V6ONLY,1) worked kadmind: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 10: udp ::.464 (pktinfo) Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 10: udp ::.464 (pktinfo) kadmind: setsockopt(11,IPV6_V6ONLY,1) worked kadmind: setsockopt(11,IPV6_V6ONLY,1) worked Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 12: tcp 0.0.0.0.464 Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 12: tcp 0.0.0.0.464 Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 11: tcp ::.464 Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 11: tcp ::.464 Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 13: rpc 0.0.0.0.88 Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 13: rpc 0.0.0.0.88 kadmind: setsockopt(14,IPV6_V6ONLY,1) worked kadmind: setsockopt(14,IPV6_V6ONLY,1) worked Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 14: rpc ::.88 Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 14: rpc ::.88 Nov 07 09:11:13 master-1 kadmind[1091](info): set up 6 sockets Nov 07 09:11:13 master-1 kadmind[1091](info): set up 6 sockets Nov 07 09:11:13 master-1 kadmind[1129](info): Seeding random number generator Nov 07 09:11:13 master-1 kadmind[1129](info): Seeding random number generator Nov 07 09:11:13 master-1 kadmind[1129](info): starting Nov 07 09:11:13 master-1 kadmind[1129](info): starting Nov 07 09:14:46 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6 Nov 07 09:14:46 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6 Nov 07 09:14:52 master-1 kadmind[1129](info): closing down fd 23 Nov 07 09:14:52 master-1 kadmind[1129](info): closing down fd 23 Nov 07 09:35:09 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6 Nov 07 09:35:09 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6 Nov 07 09:35:12 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196 Nov 07 09:35:12 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196 Nov 07 09:35:49 master-1 kadmind[1129](Notice): Request: kadm5_delete_principal, cloudera-scm/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196 Nov 07 09:35:49 master-1 kadmind[1129](Notice): Request: kadm5_delete_principal, cloudera-scm/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196 Nov 07 09:36:01 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196 Nov 07 09:36:01 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196 Nov 07 09:36:21 master-1 kadmind[1129](info): closing down fd 23 Nov 07 09:36:21 master-1 kadmind[1129](info): closing down fd 23

krb5kdc.log

Nov 06 09:47:40 master-1 krb5kdc[31776](info): setting up network... Nov 06 09:47:40 master-1 krb5kdc[31776](info): setting up network... Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo) Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 11: udp ::.88 (pktinfo) Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 11: udp ::.88 (pktinfo) krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 12: udp ::.750 (pktinfo) Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 12: udp ::.750 (pktinfo) Nov 06 09:47:40 master-1 krb5kdc[31776](info): set up 4 sockets Nov 06 09:47:40 master-1 krb5kdc[31776](info): set up 4 sockets Nov 06 09:47:40 master-1 krb5kdc[31777](info): commencing operation Nov 06 09:47:40 master-1 krb5kdc[31777](info): commencing operation Nov 06 09:52:39 master-1 krb5kdc[31777](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573033959, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 09:52:39 master-1 krb5kdc[31777](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573033959, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 12:09:18 master-1 krb5kdc[31777](debug): Got signal to request exit Nov 06 12:09:18 master-1 krb5kdc[31777](debug): Got signal to request exit Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 12 Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 12 Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 11 Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 11 Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 10 Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 10 Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 9 Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 9 Nov 06 12:09:18 master-1 krb5kdc[31777](info): shutting down Nov 06 12:09:18 master-1 krb5kdc[31777](info): shutting down Nov 06 12:37:50 master-1 krb5kdc[1081](info): setting up network... Nov 06 12:37:50 master-1 krb5kdc[1081](info): setting up network... Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo) Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 11: udp ::.88 (pktinfo) Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 11: udp ::.88 (pktinfo) krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 12: udp ::.750 (pktinfo) Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 12: udp ::.750 (pktinfo) Nov 06 12:37:50 master-1 krb5kdc[1081](info): set up 4 sockets Nov 06 12:37:50 master-1 krb5kdc[1081](info): set up 4 sockets Nov 06 12:37:50 master-1 krb5kdc[1082](info): commencing operation Nov 06 12:37:50 master-1 krb5kdc[1082](info): commencing operation Nov 06 14:29:59 master-1 krb5kdc[1082](debug): Got signal to request exit Nov 06 14:29:59 master-1 krb5kdc[1082](debug): Got signal to request exit Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 12 Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 12 Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 11 Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 11 Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 10 Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 10 Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 9 Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 9 Nov 06 14:29:59 master-1 krb5kdc[1082](info): shutting down Nov 06 14:29:59 master-1 krb5kdc[1082](info): shutting down Nov 06 14:29:59 master-1 krb5kdc[15418](info): setting up network... Nov 06 14:29:59 master-1 krb5kdc[15418](info): setting up network... Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:29:59 master-1 krb5kdc[15418](info): set up 2 sockets Nov 06 14:29:59 master-1 krb5kdc[15418](info): set up 2 sockets Nov 06 14:29:59 master-1 krb5kdc[15419](info): commencing operation Nov 06 14:29:59 master-1 krb5kdc[15419](info): commencing operation Nov 06 14:30:10 master-1 krb5kdc[15419](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573050610, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 14:30:10 master-1 krb5kdc[15419](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573050610, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 14:40:27 master-1 krb5kdc[15419](debug): Got signal to request exit Nov 06 14:40:27 master-1 krb5kdc[15419](debug): Got signal to request exit Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 10 Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 10 Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 9 Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 9 Nov 06 14:40:27 master-1 krb5kdc[15419](info): shutting down Nov 06 14:40:27 master-1 krb5kdc[15419](info): shutting down Nov 06 14:40:27 master-1 krb5kdc[16919](info): setting up network... Nov 06 14:40:27 master-1 krb5kdc[16919](info): setting up network... Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:40:27 master-1 krb5kdc[16919](info): set up 2 sockets Nov 06 14:40:27 master-1 krb5kdc[16919](info): set up 2 sockets Nov 06 14:40:27 master-1 krb5kdc[16920](info): commencing operation Nov 06 14:40:27 master-1 krb5kdc[16920](info): commencing operation Nov 06 14:44:17 master-1 krb5kdc[16920](debug): Got signal to request exit Nov 06 14:44:17 master-1 krb5kdc[16920](debug): Got signal to request exit Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 10 Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 10 Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 9 Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 9 Nov 06 14:44:17 master-1 krb5kdc[16920](info): shutting down Nov 06 14:44:17 master-1 krb5kdc[16920](info): shutting down Nov 06 14:44:17 master-1 krb5kdc[17405](info): setting up network... Nov 06 14:44:17 master-1 krb5kdc[17405](info): setting up network... Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:44:17 master-1 krb5kdc[17405](info): set up 2 sockets Nov 06 14:44:17 master-1 krb5kdc[17405](info): set up 2 sockets Nov 06 14:44:17 master-1 krb5kdc[17406](info): commencing operation Nov 06 14:44:17 master-1 krb5kdc[17406](info): commencing operation Nov 06 14:48:42 master-1 krb5kdc[17406](debug): Got signal to request exit Nov 06 14:48:42 master-1 krb5kdc[17406](debug): Got signal to request exit Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 10 Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 10 Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 9 Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 9 Nov 06 14:48:42 master-1 krb5kdc[17406](info): shutting down Nov 06 14:48:42 master-1 krb5kdc[17406](info): shutting down Nov 06 14:49:17 master-1 krb5kdc[18050](info): setting up network... Nov 06 14:49:17 master-1 krb5kdc[18050](info): setting up network... Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:49:17 master-1 krb5kdc[18050](info): set up 2 sockets Nov 06 14:49:17 master-1 krb5kdc[18050](info): set up 2 sockets Nov 06 14:49:17 master-1 krb5kdc[18051](info): commencing operation Nov 06 14:49:17 master-1 krb5kdc[18051](info): commencing operation Nov 06 14:56:50 master-1 krb5kdc[18051](debug): Got signal to request exit Nov 06 14:56:50 master-1 krb5kdc[18051](debug): Got signal to request exit Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 10 Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 10 Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 9 Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 9 Nov 06 14:56:50 master-1 krb5kdc[18051](info): shutting down Nov 06 14:56:50 master-1 krb5kdc[18051](info): shutting down Nov 06 14:56:50 master-1 krb5kdc[18977](info): setting up network... Nov 06 14:56:50 master-1 krb5kdc[18977](info): setting up network... Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 11: tcp ::.88 Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 11: tcp ::.88 Nov 06 14:56:50 master-1 krb5kdc[18977](info): set up 4 sockets Nov 06 14:56:50 master-1 krb5kdc[18977](info): set up 4 sockets Nov 06 14:56:50 master-1 krb5kdc[18978](info): commencing operation Nov 06 14:56:50 master-1 krb5kdc[18978](info): commencing operation Nov 06 15:02:32 master-1 krb5kdc[18978](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573052552, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 15:02:32 master-1 krb5kdc[18978](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573052552, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 15:06:25 master-1 krb5kdc[18978](debug): Got signal to request exit Nov 06 15:06:25 master-1 krb5kdc[18978](debug): Got signal to request exit Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 11 Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 11 Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 12 Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 12 Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 10 Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 10 Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 9 Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 9 Nov 06 15:06:25 master-1 krb5kdc[18978](info): shutting down Nov 06 15:06:25 master-1 krb5kdc[18978](info): shutting down Nov 06 15:06:25 master-1 krb5kdc[20202](info): setting up network... Nov 06 15:06:25 master-1 krb5kdc[20202](info): setting up network... Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 11: tcp ::.88 Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 11: tcp ::.88 Nov 06 15:06:25 master-1 krb5kdc[20202](info): set up 4 sockets Nov 06 15:06:25 master-1 krb5kdc[20202](info): set up 4 sockets Nov 06 15:06:25 master-1 krb5kdc[20203](info): commencing operation Nov 06 15:06:25 master-1 krb5kdc[20203](info): commencing operation Nov 06 15:09:45 master-1 krb5kdc[20203](debug): Got signal to request exit Nov 06 15:09:45 master-1 krb5kdc[20203](debug): Got signal to request exit Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 11 Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 11 Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 12 Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 12 Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 10 Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 10 Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 9 Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 9 Nov 06 15:09:45 master-1 krb5kdc[20203](info): shutting down Nov 06 15:09:45 master-1 krb5kdc[20203](info): shutting down Nov 06 15:09:45 master-1 krb5kdc[20588](info): setting up network... Nov 06 15:09:45 master-1 krb5kdc[20588](info): setting up network... Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 11: tcp ::.88 Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 11: tcp ::.88 Nov 06 15:09:45 master-1 krb5kdc[20588](info): set up 4 sockets Nov 06 15:09:45 master-1 krb5kdc[20588](info): set up 4 sockets Nov 06 15:09:45 master-1 krb5kdc[20589](info): commencing operation Nov 06 15:09:45 master-1 krb5kdc[20589](info): commencing operation Nov 06 15:10:23 master-1 krb5kdc[20589](debug): Got signal to request exit Nov 06 15:10:23 master-1 krb5kdc[20589](debug): Got signal to request exit Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 11 Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 11 Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 12 Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 12 Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 10 Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 10 Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 9 Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 9 Nov 06 15:10:23 master-1 krb5kdc[20589](info): shutting down Nov 06 15:10:23 master-1 krb5kdc[20589](info): shutting down Nov 06 15:10:23 master-1 krb5kdc[20692](info): setting up network... Nov 06 15:10:23 master-1 krb5kdc[20692](info): setting up network... Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 11: tcp ::.88 Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 11: tcp ::.88 Nov 06 15:10:23 master-1 krb5kdc[20692](info): set up 4 sockets Nov 06 15:10:23 master-1 krb5kdc[20692](info): set up 4 sockets Nov 06 15:10:23 master-1 krb5kdc[20693](info): commencing operation Nov 06 15:10:23 master-1 krb5kdc[20693](info): commencing operation Nov 06 15:10:25 master-1 krb5kdc[20693](info): AS_REQ (1 etypes {17}) 10.243.1.196: ISSUE: authtime 1573053025, etypes {rep=17 tkt=17 ses=17}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 15:10:25 master-1 krb5kdc[20693](info): AS_REQ (1 etypes {17}) 10.243.1.196: ISSUE: authtime 1573053025, etypes {rep=17 tkt=17 ses=17}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 15:10:25 master-1 krb5kdc[20693](info): closing down fd 13 Nov 06 15:10:25 master-1 krb5kdc[20693](info): closing down fd 13 Nov 06 15:12:17 master-1 krb5kdc[20693](debug): Got signal to request exit Nov 06 15:12:17 master-1 krb5kdc[20693](debug): Got signal to request exit Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 11 Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 11 Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 12 Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 12 Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 10 Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 10 Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 9 Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 9 Nov 06 15:12:17 master-1 krb5kdc[20693](info): shutting down Nov 06 15:12:17 master-1 krb5kdc[20693](info): shutting down Nov 06 15:12:17 master-1 krb5kdc[20926](info): setting up network... Nov 06 15:12:17 master-1 krb5kdc[20926](info): setting up network... Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 11: tcp ::.88 Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 11: tcp ::.88 Nov 06 15:12:17 master-1 krb5kdc[20926](info): set up 4 sockets Nov 06 15:12:17 master-1 krb5kdc[20926](info): set up 4 sockets Nov 06 15:12:17 master-1 krb5kdc[20927](info): commencing operation Nov 06 15:12:17 master-1 krb5kdc[20927](info): commencing operation Nov 06 15:12:49 master-1 krb5kdc[20927](debug): Got signal to request exit Nov 06 15:12:49 master-1 krb5kdc[20927](debug): Got signal to request exit Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 11 Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 11 Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 12 Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 12 Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 10 Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 10 Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 9 Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 9 Nov 06 15:12:49 master-1 krb5kdc[20927](info): shutting down Nov 06 15:12:49 master-1 krb5kdc[20927](info): shutting down Nov 06 15:12:49 master-1 krb5kdc[20993](info): setting up network... Nov 06 15:12:49 master-1 krb5kdc[20993](info): setting up network... Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 11: tcp ::.88 Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 11: tcp ::.88 Nov 06 15:12:49 master-1 krb5kdc[20993](info): set up 4 sockets Nov 06 15:12:49 master-1 krb5kdc[20993](info): set up 4 sockets Nov 06 15:12:49 master-1 krb5kdc[20994](info): commencing operation Nov 06 15:12:49 master-1 krb5kdc[20994](info): commencing operation Nov 06 15:14:48 master-1 krb5kdc[20994](debug): Got signal to request exit Nov 06 15:14:48 master-1 krb5kdc[20994](debug): Got signal to request exit Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 11 Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 11 Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 12 Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 12 Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 10 Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 10 Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 9 Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 9 Nov 06 15:14:48 master-1 krb5kdc[20994](info): shutting down Nov 06 15:14:48 master-1 krb5kdc[20994](info): shutting down krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm Nov 06 15:16:56 master-1 krb5kdc[21561](info): setting up network... Nov 06 15:16:56 master-1 krb5kdc[21561](info): setting up network... Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 11: tcp ::.88 Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 11: tcp ::.88 Nov 06 15:16:56 master-1 krb5kdc[21561](info): set up 4 sockets Nov 06 15:16:56 master-1 krb5kdc[21561](info): set up 4 sockets Nov 06 15:16:56 master-1 krb5kdc[21562](info): commencing operation Nov 06 15:16:56 master-1 krb5kdc[21562](info): commencing operation Nov 06 17:45:15 master-1 krb5kdc[21562](debug): Got signal to request exit Nov 06 17:45:15 master-1 krb5kdc[21562](debug): Got signal to request exit Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 11 Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 11 Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 12 Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 12 Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 10 Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 10 Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 9 Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 9 Nov 06 17:45:15 master-1 krb5kdc[21562](info): shutting down Nov 06 17:45:15 master-1 krb5kdc[21562](info): shutting down Nov 06 17:45:15 master-1 krb5kdc[8642](info): setting up network... Nov 06 17:45:15 master-1 krb5kdc[8642](info): setting up network... Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 10: udp ::.88 (pktinfo) Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88 Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation Nov 06 17:58:08 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573063088, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 06 17:58:08 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573063088, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 07 09:03:31 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573117411, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM Nov 07 09:03:31 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573117411, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM Nov 07 09:03:51 master-1 krb5kdc[8643](Error): TCP client 10.243.1.196.60500 wants 2147484324 bytes, cap is 1048572 Nov 07 09:03:51 master-1 krb5kdc[8643](Error): TCP client 10.243.1.196.60500 wants 2147484324 bytes, cap is 1048572 Nov 07 09:03:51 master-1 krb5kdc[8643](info): closing down fd 13 Nov 07 09:03:51 master-1 krb5kdc[8643](info): closing down fd 13 Nov 07 09:05:36 master-1 krb5kdc[8643](debug): Got signal to request exit Nov 07 09:05:36 master-1 krb5kdc[8643](debug): Got signal to request exit Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 11 Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 11 Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 12 Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 12 Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 10 Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 10 Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 9 Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 9 Nov 07 09:05:36 master-1 krb5kdc[8643](info): shutting down Nov 07 09:05:36 master-1 krb5kdc[8643](info): shutting down Nov 07 09:05:36 master-1 krb5kdc[29966](info): setting up network... Nov 07 09:05:36 master-1 krb5kdc[29966](info): setting up network... Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 10: udp ::.88 (pktinfo) Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 12: tcp 0.0.0.0.88 Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 12: tcp 0.0.0.0.88 Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 11: tcp ::.88 Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 11: tcp ::.88 Nov 07 09:05:36 master-1 krb5kdc[29966](info): set up 4 sockets Nov 07 09:05:36 master-1 krb5kdc[29966](info): set up 4 sockets Nov 07 09:05:36 master-1 krb5kdc[29967](info): commencing operation Nov 07 09:05:36 master-1 krb5kdc[29967](info): commencing operation Nov 07 09:10:10 master-1 krb5kdc[29967](debug): Got signal to request exit Nov 07 09:10:10 master-1 krb5kdc[29967](debug): Got signal to request exit Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 11 Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 11 Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 12 Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 12 Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 10 Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 10 Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 9 Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 9 Nov 07 09:10:10 master-1 krb5kdc[29967](info): shutting down Nov 07 09:10:10 master-1 krb5kdc[29967](info): shutting down Nov 07 09:11:13 master-1 krb5kdc[1093](info): setting up network... Nov 07 09:11:13 master-1 krb5kdc[1093](info): setting up network... Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 10: udp ::.88 (pktinfo) Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc: Address already in use - Cannot bind server socket on ::.88 Nov 07 09:11:13 master-1 krb5kdc[1093](info): set up 2 sockets Nov 07 09:11:13 master-1 krb5kdc[1093](info): set up 2 sockets Nov 07 09:11:13 master-1 krb5kdc[1130](info): commencing operation Nov 07 09:11:13 master-1 krb5kdc[1130](info): commencing operation Nov 07 09:14:09 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118049, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM Nov 07 09:14:09 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118049, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM Nov 07 09:16:38 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118198, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 07 09:16:38 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118198, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 07 09:34:53 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119293, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM Nov 07 09:34:53 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119293, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM Nov 07 09:37:40 master-1 krb5kdc[1130](debug): Got signal to request exit Nov 07 09:37:40 master-1 krb5kdc[1130](debug): Got signal to request exit Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 10 Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 10 Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 9 Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 9 Nov 07 09:37:40 master-1 krb5kdc[1130](info): shutting down Nov 07 09:37:40 master-1 krb5kdc[1130](info): shutting down Nov 07 09:37:40 master-1 krb5kdc[8394](info): setting up network... Nov 07 09:37:40 master-1 krb5kdc[8394](info): setting up network... Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 10: udp ::.88 (pktinfo) Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc: Address already in use - Cannot bind server socket on ::.88 Nov 07 09:37:40 master-1 krb5kdc[8394](info): set up 2 sockets Nov 07 09:37:40 master-1 krb5kdc[8394](info): set up 2 sockets Nov 07 09:37:40 master-1 krb5kdc[8396](info): commencing operation Nov 07 09:37:40 master-1 krb5kdc[8396](info): commencing operation Nov 07 09:37:55 master-1 krb5kdc[8396](debug): Got signal to request exit Nov 07 09:37:55 master-1 krb5kdc[8396](debug): Got signal to request exit Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 10 Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 10 Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 9 Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 9 Nov 07 09:37:55 master-1 krb5kdc[8396](info): shutting down Nov 07 09:37:55 master-1 krb5kdc[8396](info): shutting down Nov 07 09:38:00 master-1 krb5kdc[8468](info): setting up network... Nov 07 09:38:00 master-1 krb5kdc[8468](info): setting up network... Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 10: udp ::.88 (pktinfo) Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc: Address already in use - Cannot bind server socket on ::.88 Nov 07 09:38:00 master-1 krb5kdc[8468](info): set up 2 sockets Nov 07 09:38:00 master-1 krb5kdc[8468](info): set up 2 sockets Nov 07 09:38:00 master-1 krb5kdc[8469](info): commencing operation Nov 07 09:38:00 master-1 krb5kdc[8469](info): commencing operation Nov 07 09:38:08 master-1 krb5kdc[8469](debug): Got signal to request exit Nov 07 09:38:08 master-1 krb5kdc[8469](debug): Got signal to request exit Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 10 Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 10 Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 9 Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 9 Nov 07 09:38:08 master-1 krb5kdc[8469](info): shutting down Nov 07 09:38:08 master-1 krb5kdc[8469](info): shutting down Nov 07 09:42:37 master-1 krb5kdc[9029](info): setting up network... Nov 07 09:42:37 master-1 krb5kdc[9029](info): setting up network... Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 10: udp ::.88 (pktinfo) Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc: Address already in use - Cannot bind server socket on ::.88 Nov 07 09:42:37 master-1 krb5kdc[9029](info): set up 2 sockets Nov 07 09:42:37 master-1 krb5kdc[9029](info): set up 2 sockets Nov 07 09:42:37 master-1 krb5kdc[9030](info): commencing operation Nov 07 09:42:37 master-1 krb5kdc[9030](info): commencing operation Nov 07 09:42:55 master-1 krb5kdc[9030](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119775, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 07 09:42:55 master-1 krb5kdc[9030](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119775, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 07 12:32:38 master-1 krb5kdc[9030](debug): Got signal to request exit Nov 07 12:32:38 master-1 krb5kdc[9030](debug): Got signal to request exit Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 10 Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 10 Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 9 Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 9 Nov 07 12:32:38 master-1 krb5kdc[9030](info): shutting down Nov 07 12:32:38 master-1 krb5kdc[9030](info): shutting down Nov 07 12:32:42 master-1 krb5kdc[30870](info): setting up network... Nov 07 12:32:42 master-1 krb5kdc[30870](info): setting up network... Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 10: udp ::.88 (pktinfo) Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc: Address already in use - Cannot bind server socket on ::.88 Nov 07 12:32:42 master-1 krb5kdc[30870](info): set up 2 sockets Nov 07 12:32:42 master-1 krb5kdc[30870](info): set up 2 sockets Nov 07 12:32:42 master-1 krb5kdc[30871](info): commencing operation Nov 07 12:32:42 master-1 krb5kdc[30871](info): commencing operation Nov 07 12:40:14 master-1 krb5kdc[30871](debug): Got signal to request exit Nov 07 12:40:14 master-1 krb5kdc[30871](debug): Got signal to request exit Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 10 Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 10 Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 9 Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 9 Nov 07 12:40:14 master-1 krb5kdc[30871](info): shutting down Nov 07 12:40:14 master-1 krb5kdc[30871](info): shutting down Nov 07 12:40:19 master-1 krb5kdc[31877](info): setting up network... Nov 07 12:40:19 master-1 krb5kdc[31877](info): setting up network... Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo) Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc: Address already in use - Cannot bind server socket on ::.88 Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets Nov 07 12:40:19 master-1 krb5kdc[31878](info): commencing operation Nov 07 12:40:19 master-1 krb5kdc[31878](info): commencing operation Nov 07 13:05:51 master-1 krb5kdc[31878](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573131951, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM Nov 07 13:05:51 master-1 krb5kdc[31878](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573131951, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM

Thanks

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

Shelton — Thu, 07 Nov 2019 15:18:10 GMT

@svasi

For the logs I can see that the UDP port 88 used by the KDC is already in use by some application.

Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc:
Address already in use - Cannot bind server socket on ::.88
Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets
Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets

The default ports used by Kerberos are port 88 for the KDC1 and port 749 for the admin server. You can, however, choose to run on other ports, as long as they are specified in each host's /etc/services and krb5.conf files, and the kdc.conf file on each KDC

Solution one

Check for the PID of port 88
# netstat -tupln

kill PID
# kill -9 <PID>

Retry your kerberiszation

Solution 2

Change the port to something like 8888 in /etc/services and krb5.conf files, and the kdc.conf

Restart the kdc and kadmin and retry the kerberization

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

Shelton — Fri, 08 Nov 2019 05:16:26 GMT

@svasi

Any feedback on this issue?

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Fri, 08 Nov 2019 09:37:29 GMT

I've solved the UDP port 88 problem it was a dead kadmid process.

I've killed it and restarted Kerberos services.

Now both krb5kdc and kadmind are up without issues.

Here their log files

krb5kd5.log

Nov 08 07:35:03 master-1 krb5kdc[22649](info): setting up network... Nov 08 07:35:03 master-1 krb5kdc[22649](info): setting up network... Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 10: udp ::.88 (pktinfo) Nov 08 07:35:03 master-1 krb5kdc[22649](info): listening on fd 10: udp ::.88 (pktinfo) Nov 08 07:35:03 master-1 krb5kdc[22649](info): set up 2 sockets Nov 08 07:35:03 master-1 krb5kdc[22649](info): set up 2 sockets Nov 08 07:35:03 master-1 krb5kdc[22650](info): commencing operation Nov 08 07:35:03 master-1 krb5kdc[22650](info): commencing operation

kadmind.log

Nov 08 07:35:26 master-1 kadmind[22689](info): setting up network... Nov 08 07:35:26 master-1 kadmind[22689](info): setting up network... Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo) Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo) kadmind: setsockopt(10,IPV6_V6ONLY,1) worked kadmind: setsockopt(10,IPV6_V6ONLY,1) worked Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 10: udp ::.464 (pktinfo) Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 10: udp ::.464 (pktinfo) kadmind: setsockopt(11,IPV6_V6ONLY,1) worked kadmind: setsockopt(11,IPV6_V6ONLY,1) worked Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 12: tcp 0.0.0.0.464 Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 12: tcp 0.0.0.0.464 Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 11: tcp ::.464 Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 11: tcp ::.464 Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 13: rpc 0.0.0.0.749 Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 13: rpc 0.0.0.0.749 kadmind: setsockopt(14,IPV6_V6ONLY,1) worked kadmind: setsockopt(14,IPV6_V6ONLY,1) worked Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 14: rpc ::.749 Nov 08 07:35:26 master-1 kadmind[22689](info): listening on fd 14: rpc ::.749 Nov 08 07:35:26 master-1 kadmind[22689](info): set up 6 sockets Nov 08 07:35:26 master-1 kadmind[22689](info): set up 6 sockets Nov 08 07:35:26 master-1 kadmind[22690](info): Seeding random number generator Nov 08 07:35:26 master-1 kadmind[22690](info): Seeding random number generator Nov 08 07:35:26 master-1 kadmind[22690](info): starting Nov 08 07:35:26 master-1 kadmind[22690](info): starting

Socket status

master-1:~ # netstat -tupln|grep 749 tcp 0 0 0.0.0.0:749 0.0.0.0:* LISTEN 22452/kadmind tcp 0 0 :::749 :::* LISTEN 22452/kadmind master-1:~ # netstat -tupln|grep 88 tcp 0 0 10.243.1.196:8088 0.0.0.0:* LISTEN 2050/java tcp 0 0 0.0.0.0:18088 0.0.0.0:* LISTEN 2048/java tcp 0 0 10.243.1.196:19888 0.0.0.0:* LISTEN 2461/java udp 0 0 0.0.0.0:88 0.0.0.0:* 22439/krb5kdc udp 0 0 :::88 :::* 22439/krb5kdc

Anyway I still can't proceed with Cloudera kerberization because when I push (several times) the CONTINUE button in the right lower corner, which is enabled, nothing happens and the wizard remains on the Setup KDC Account page.

Here's a screenshot from the wizard where I try to click, even several times, but nothing happens.

I've tried with Mozilla, Edge, Chrome and IE browsers with no luck.

If I change the principal realm in the wizard, cloudera manager raises a warning (see below picture) and the CONTINUE button is disabled.

So it doesn't seem to be a Kerberos related issue.

Cloudera manager seems aware of Kerberos configuration.

From shell I can work on principals, adding or deleting, obtain tickets with kinit and destroy them with kdestroy.

I've set Cloudera Manager logs at DEBUG level but nothing is traced, neither in the above kerberos log files, when I click the above button.

Really can't figure out where the problem is.

Any idea?

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Fri, 08 Nov 2019 08:34:39 GMT

I'm reading again Kerberos docs and I had doubt on my krb5.conf domain realm section.

My network domain is dev.edl.gcp.domain.it so I've added its realm translation but still not working in cloudera manager.

[domain_realm] .edhdev.com = EDHDEV.COM edhdev.com = EDHDEV.COM .dev.edl.gcp.domain.it = EDHDEV.COM dev.edl.gcp.domain.it = EDHDEV.COM

So I've removed it.

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

svasi — Fri, 08 Nov 2019 16:04:37 GMT

Just tried to set up a different cluster, also on Google Cloud Platform, with a CDH 5.15 to try a new setup on another version.

Same configuration, same behaviour.

Another setup local to my datacenter has no problem.

Does anybody met same problem on GCP?

Re: Enable Kerberos - Setup KDC Account - Continue enabled but not working

Shelton — Fri, 08 Nov 2019 17:26:50 GMT

@svasi

WOW we have been going around for some time now, you realize its always very important to give a comprehensive description of your environment this helps quickly zero on the problem.

I have never deployed on GCP so I can't be of much help on the platform side.

Below is a document you should have gone through to give you pointers and avoid the frustration 🙂

https://docs.cloudera.com/documentation/director/latest/topics/director_get_started_gcp_install_cm_cdh.html

Please read this document and revert