Support Questions

cjervis · ‎11-06-2019

Hi.

I'm kerberizing an EDH 6.1.1 on SLES on Google GCE.

Kerberos is up and running.

I've created a cloudera-scm/admin@REALM and gained its ticket with kinit, so I suppose everything is fine.

I try to enable Kerberos using the CM wizard.

When I reach the Setup KDC Account I set the cloudera-scm@REALM principal and the Continue button is enabled.

I try to click on it to proceed but nothing happens.

I took a look at krb5 log files and to cloudera-scm-server log files but nothing seems to be logged.

I've doublechecked NTP, Reverse DNS and krb5.conf file but found nothing useful.

Any help ?

Thanks

Stefano

svasi · ‎01-24-2020

Hi @Shelton

Finally I've solved the problem and hope my experience could help someone else who is banging his/her head on this scenario.

At time of writing I was accessing the GCP cluster via public IP addresses.

I had a /etc/hosts file resolving hostnames to their public IP

Then customer's NOC granted me access from internal network over VPN and next attempt went fine.

Still can't understand why, maybe some networking related config issue, but being on the internal side of the network solved my problem

Simple as that!

Thanks for your precious support, and patience

Stefano

View solution in original post

Shelton · ‎11-06-2019

@svasi

Can you explain one cannot guess with the information you have given.

HDP or CDH?

OS

Steps executed

Krb5.conf

Kdc.conf

kadm5.acl

Error log or screenshot.

With the above the problem could be easier to analyze

svasi · ‎11-06-2019

Thanks @Shelton

Is a CDH 6.1.1 on SUSE Enterprise 12.4 running on Google GCP compute engines.

I've installed, with zypper, those packages on the CM server:

zypper install krb5-server krb5-client krb5 openldap2-client

This is the /etc/krb5.conf file

[libdefaults]
    # "dns_canonicalize_hostname" and "rdns" are better set to false for improved security.
    # If set to true, the canonicalization mechanism performed by Kerberos client may
    # allow service impersonification, the consequence is similar to conducting TLS certificate
    # verification without checking host name.
    # If left unspecified, the two parameters will have default value true, which is less secure.
    dns_canonicalize_hostname = false
    rdns = false
    default_realm = EDHDEV.COM
    default_ccache_name = FILE:/tmp/krb5cc_%{uid}

[realms]
        EDHDEV.COM = {
                kdc = master-1.dev.edl.gcp.domain.it
                admin_server = master-1.dev.edl.gcp.domain.it
        }

[logging]
    kdc = FILE:/var/log/krb5/krb5kdc.log
    admin_server = FILE:/var/log/krb5/kadmind.log
    default = SYSLOG:NOTICE:DAEMON

This is the kdc.conf file

[kdcdefaults]
        kdc_ports = 88
        kdc_tcp_ports = 88

[realms]
        EDHDEV.COM = {
                database_name = /var/lib/kerberos/krb5kdc/principal
                admin_keytab = FILE:/var/lib/kerberos/krb5kdc/kadm5.keytab
                acl_file = /var/lib/kerberos/krb5kdc/kadm5.acl
                dict_file = /var/lib/kerberos/krb5kdc/kadm5.dict
                key_stash_file = /var/lib/kerberos/krb5kdc/.k5.EDHDEV.COM
                kdc_ports = 88
                max_life = 1d
                max_renewable_life = 7d
        }

[logging]
    kdc = FILE:/var/log/krb5/krb5kdc.log
    admin_server = FILE:/var/log/krb5/kadmind.log

This is the kadm5.acl file:

###############################################################################
#Kerberos_principal      permissions     [target_principal]      [restrictions]
###############################################################################
#
*/admin@EDHDEV.COM  *

I've created, using kadmin.local, the cloudera-scm/admin@EDHDEV.COM

master-1:~ # kadmin.local
Authenticating as principal cloudera-scm/admin@EDHDEV.COM with password.
kadmin.local:  listprincs
K/M@EDHDEV.COM
cloudera-scm/admin@EDHDEV.COM
kadmin/admin@EDHDEV.COM
kadmin/changepw@EDHDEV.COM
kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
krbtgt/EDHDEV.COM@EDHDEV.COM

I can authenticate against kerberos with kinit and gain the ticket (It takes arount 8 seconds to get the ticket):

master-1:~ # kinit cloudera-scm/admin@EDHDEV.COM
Password for cloudera-scm/admin@EDHDEV.COM:
master-1:~ # klist -e
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: cloudera-scm/admin@EDHDEV.COM

Valid starting     Expires            Service principal
11/06/19 17:58:08  11/07/19 17:58:08  krbtgt/EDHDEV.COM@EDHDEV.COM
        Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

I run the wizard in CM to enable Kerberos on my cluster.

I set KDC host and Encryption type according to previous output

Here I configure the principal but nothing happens when I click on Continue:

Kerberos log files:

==> /var/log/krb5/kadmind.log <==
Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets
Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets
Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator
Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator
Nov 06 12:37:51 master-1 kadmind[1109](info): starting
Nov 06 12:37:51 master-1 kadmind[1109](info): starting
Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one.
Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one.
Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one.
Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one.

==> /var/log/krb5/krb5kdc.log <==
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets
Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets
Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation
Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation

cloudera-scm-server.log when I change from manage krb5.conf page to Setup KDC Account one and, then, select Continue button

2019-11-06 17:49:40,620 INFO scm-web-154:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/v15/batch
2019-11-06 17:49:40,625 INFO scm-web-154:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/v15/batch, Status:200
2019-11-06 17:49:40,885 INFO scm-web-380:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/v15/batch
2019-11-06 17:49:40,889 INFO scm-web-380:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/v15/batch, Status:200

Shelton · ‎11-06-2019

@svasi

I saw a couple of potential issues and I would like you to make a few changes.

Recreate an admin for Kerberos

Using the below syntax

# kadmin.local -q "addprinc admin/admin"

----------start of output---------
Authenticating as principal root/admin@EDHDEV.COM with a password.
WARNING: no policy specified for admin/admin@EDHDEV.COM; defaulting to no policy
Enter the password for principal "admin/admin@EDHDEV.COM": welcome1
Re-enter password for principal "admin/admin@EDHDEV.COM": welcome1
Principal "admin/admin@EDHDEV.COM" created.
----------end of output---------

Validate the principal works

# kadmin -p admin/admin@EDHDEV.COM

It will ask for the password once successfully authenticated then type q to quit

Can you leave the default encryption types? and you should select Manage krb5.conf through Cloudera Manager

Don't change the kadm5.acl file it's okay the way it is !!

I saw something missing in your krb5.conf between [realms] and [logging] the domain_realm add that to all the hosts if it's a multinode

[realms]
EDHDEV.COM = {
kdc = master-1.dev.edl.gcp.domain.it
admin_server = master-1.dev.edl.gcp.domain.it
}

[domain_realm]
.edhdev.com = EDHDEV.COM
edhdev.com = EDHDEV.COM

[logging]
kdc = FILE:/var/log/krb5/krb5kdc.log
admin_server = FILE:/var/log/krb5/kadmind.log
default = SYSLOG:NOTICE:DAEMON

Go to the CM and launch the kerberization using this below the principal

admin/admin@EDHDEV.COM

Please make the above changes restart the kdc and retry kerberization it should go smoothly

Happy hadooping

svasi · ‎11-07-2019

Thanks @Shelton

Just followed your post but I still can't continue over Setup KDC Account step.

##### CREATED ADMIN PRINCIPAL

master-1:~ # kadmin.local -q "addprinc admin/admin"
Authenticating as principal cloudera-scm/admin@EDHDEV.COM with password.
WARNING: no policy specified for admin/admin@EDHDEV.COM; defaulting to no policy
Enter password for principal "admin/admin@EDHDEV.COM":
Re-enter password for principal "admin/admin@EDHDEV.COM":
Principal "admin/admin@EDHDEV.COM" created.

.........................................................................

##### VALIDATED ADMIN PRINCIPAL

master-1:~ # kadmin -p admin/admin@EDHDEV.COM
Authenticating as principal admin/admin@EDHDEV.COM with password.
Password for admin/admin@EDHDEV.COM:
kadmin:  q

Continue button is enabled but pressing it does nothing and procedure remains on the same page.

Nothing is written in krb5 or cloudera manager log files.

I've raised cloudera manager's log4j level to DEBUG but still nothing is traced when I press Continue button

Shelton · ‎11-07-2019

@svasi

Can you share the files mentioned in my previous post? Redact site specific sensitive information and I will have a look at it.

svasi · ‎11-07-2019

krb5.conf

[libdefaults]
    # "dns_canonicalize_hostname" and "rdns" are better set to false for improved security.
    # If set to true, the canonicalization mechanism performed by Kerberos client may
    # allow service impersonification, the consequence is similar to conducting TLS certificate
    # verification without checking host name.
    # If left unspecified, the two parameters will have default value true, which is less secure.
    dns_canonicalize_hostname = false
    rdns = false
    default_realm = EDHDEV.COM
    default_ccache_name = FILE:/tmp/krb5cc_%{uid}

[realms]
        EDHDEV.COM = {
                kdc = master-1.dev.edl.gcp.domain.it
                admin_server = master-1.dev.edl.gcp.domain.it
        }


[domain_realm]
        .edhdev.com = EDHDEV.COM
        edhdev.com = EDHDEV.COM

[logging]
    kdc = FILE:/var/log/krb5/krb5kdc.log
    admin_server = FILE:/var/log/krb5/kadmind.log
    default = SYSLOG:NOTICE:DAEMON

kdc.conf

[kdcdefaults]
        kdc_ports = 88
        kdc_tcp_ports = 88

[realms]
        EDHDEV.COM = {
                database_name = /var/lib/kerberos/krb5kdc/principal
                admin_keytab = FILE:/var/lib/kerberos/krb5kdc/kadm5.keytab
                acl_file = /var/lib/kerberos/krb5kdc/kadm5.acl
                dict_file = /var/lib/kerberos/krb5kdc/kadm5.dict
                key_stash_file = /var/lib/kerberos/krb5kdc/.k5.EDHDEV.COM
                kdc_ports = 88
                max_life = 1d
                max_renewable_life = 7d
        }

[logging]
    kdc = FILE:/var/log/krb5/krb5kdc.log
    admin_server = FILE:/var/log/krb5/kadmind.log

kadm5.acl

###############################################################################
#Kerberos_principal      permissions     [target_principal]      [restrictions]
###############################################################################
#
*/admin@EDHDEV.COM  *

Shelton · ‎11-07-2019

@svasi

Sorry also these 2 logs the files attached look okay,

/var/log/krb5/krb5kdc.log
/var/log/krb5/kadmind.log

Did you create the re-create the principal? Share the output of

# kadmin.local

listprincs

svasi · ‎11-07-2019

kadmin.local

listprincs

master-1:/var/log/krb5 # kadmin.local
Authenticating as principal admin/admin@EDHDEV.COM with password.
kadmin.local:  listprincs
K/M@EDHDEV.COM
admin/admin@EDHDEV.COM
kadmin/admin@EDHDEV.COM
kadmin/changepw@EDHDEV.COM
kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
krbtgt/EDHDEV.COM@EDHDEV.COM

kadmin.log

Nov 06 09:39:36 master-1 kdb5_util[30635](info): No dictionary file specified, continuing without one.
Nov 06 09:39:36 master-1 kdb5_util[30635](info): No dictionary file specified, continuing without one.
Nov 06 09:42:18 master-1 kadmin.local[31087](info): No dictionary file specified, continuing without one.
Nov 06 09:42:18 master-1 kadmin.local[31087](info): No dictionary file specified, continuing without one.
Nov 06 12:37:51 master-1 kadmind[1083](info): setting up network...
Nov 06 12:37:51 master-1 kadmind[1083](info): setting up network...
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo)
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo)
kadmind: setsockopt(10,IPV6_V6ONLY,1) worked
kadmind: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 10: udp ::.464 (pktinfo)
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 10: udp ::.464 (pktinfo)
kadmind: setsockopt(11,IPV6_V6ONLY,1) worked
kadmind: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 12: tcp 0.0.0.0.464
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 12: tcp 0.0.0.0.464
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 11: tcp ::.464
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 11: tcp ::.464
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 13: rpc 0.0.0.0.749
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 13: rpc 0.0.0.0.749
kadmind: setsockopt(14,IPV6_V6ONLY,1) worked
kadmind: setsockopt(14,IPV6_V6ONLY,1) worked
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 14: rpc ::.749
Nov 06 12:37:51 master-1 kadmind[1083](info): listening on fd 14: rpc ::.749
Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets
Nov 06 12:37:51 master-1 kadmind[1083](info): set up 6 sockets
Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator
Nov 06 12:37:51 master-1 kadmind[1109](info): Seeding random number generator
Nov 06 12:37:51 master-1 kadmind[1109](info): starting
Nov 06 12:37:51 master-1 kadmind[1109](info): starting
Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one.
Nov 06 15:18:58 master-1 kadmin.local[21785](info): No dictionary file specified, continuing without one.
Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one.
Nov 06 15:20:01 master-1 kadmin.local[21922](info): No dictionary file specified, continuing without one.
Nov 07 09:10:10 master-1 kadmind[1109](debug): Got signal to request exit
Nov 07 09:10:10 master-1 kadmind[1109](debug): Got signal to request exit
Nov 07 09:10:10 master-1 kadmind[1109](info): finished, exiting
Nov 07 09:10:10 master-1 kadmind[1109](info): finished, exiting
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 14
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 14
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 13
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 13
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 11
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 11
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 12
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 12
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 10
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 10
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 9
Nov 07 09:10:10 master-1 kadmind[1109](info): closing down fd 9
Nov 07 09:11:13 master-1 kadmind[1091](info): setting up network...
Nov 07 09:11:13 master-1 kadmind[1091](info): setting up network...
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo)
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 9: udp 0.0.0.0.464 (pktinfo)
kadmind: setsockopt(10,IPV6_V6ONLY,1) worked
kadmind: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 10: udp ::.464 (pktinfo)
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 10: udp ::.464 (pktinfo)
kadmind: setsockopt(11,IPV6_V6ONLY,1) worked
kadmind: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 12: tcp 0.0.0.0.464
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 12: tcp 0.0.0.0.464
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 11: tcp ::.464
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 11: tcp ::.464
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 13: rpc 0.0.0.0.88
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 13: rpc 0.0.0.0.88
kadmind: setsockopt(14,IPV6_V6ONLY,1) worked
kadmind: setsockopt(14,IPV6_V6ONLY,1) worked
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 14: rpc ::.88
Nov 07 09:11:13 master-1 kadmind[1091](info): listening on fd 14: rpc ::.88
Nov 07 09:11:13 master-1 kadmind[1091](info): set up 6 sockets
Nov 07 09:11:13 master-1 kadmind[1091](info): set up 6 sockets
Nov 07 09:11:13 master-1 kadmind[1129](info): Seeding random number generator
Nov 07 09:11:13 master-1 kadmind[1129](info): Seeding random number generator
Nov 07 09:11:13 master-1 kadmind[1129](info): starting
Nov 07 09:11:13 master-1 kadmind[1129](info): starting
Nov 07 09:14:46 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6
Nov 07 09:14:46 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6
Nov 07 09:14:52 master-1 kadmind[1129](info): closing down fd 23
Nov 07 09:14:52 master-1 kadmind[1129](info): closing down fd 23
Nov 07 09:35:09 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6
Nov 07 09:35:09 master-1 kadmind[1129](Notice): Request: kadm5_init, admin/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196, vers=4, flavor=6
Nov 07 09:35:12 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196
Nov 07 09:35:12 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196
Nov 07 09:35:49 master-1 kadmind[1129](Notice): Request: kadm5_delete_principal, cloudera-scm/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196
Nov 07 09:35:49 master-1 kadmind[1129](Notice): Request: kadm5_delete_principal, cloudera-scm/admin@EDHDEV.COM, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196
Nov 07 09:36:01 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196
Nov 07 09:36:01 master-1 kadmind[1129](Notice): Request: kadm5_get_principals, *, success, client=admin/admin@EDHDEV.COM, service=kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM, addr=10.243.1.196
Nov 07 09:36:21 master-1 kadmind[1129](info): closing down fd 23
Nov 07 09:36:21 master-1 kadmind[1129](info): closing down fd 23

krb5kdc.log

Nov 06 09:47:40 master-1 krb5kdc[31776](info): setting up network...
Nov 06 09:47:40 master-1 krb5kdc[31776](info): setting up network...
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo)
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 11: udp ::.88 (pktinfo)
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 11: udp ::.88 (pktinfo)
krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 12: udp ::.750 (pktinfo)
Nov 06 09:47:40 master-1 krb5kdc[31776](info): listening on fd 12: udp ::.750 (pktinfo)
Nov 06 09:47:40 master-1 krb5kdc[31776](info): set up 4 sockets
Nov 06 09:47:40 master-1 krb5kdc[31776](info): set up 4 sockets
Nov 06 09:47:40 master-1 krb5kdc[31777](info): commencing operation
Nov 06 09:47:40 master-1 krb5kdc[31777](info): commencing operation
Nov 06 09:52:39 master-1 krb5kdc[31777](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573033959, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 09:52:39 master-1 krb5kdc[31777](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573033959, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 12:09:18 master-1 krb5kdc[31777](debug): Got signal to request exit
Nov 06 12:09:18 master-1 krb5kdc[31777](debug): Got signal to request exit
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 12
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 12
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 11
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 11
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 10
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 10
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 9
Nov 06 12:09:18 master-1 krb5kdc[31777](info): closing down fd 9
Nov 06 12:09:18 master-1 krb5kdc[31777](info): shutting down
Nov 06 12:09:18 master-1 krb5kdc[31777](info): shutting down
Nov 06 12:37:50 master-1 krb5kdc[1081](info): setting up network...
Nov 06 12:37:50 master-1 krb5kdc[1081](info): setting up network...
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo)
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 10: udp 0.0.0.0.750 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 11: udp ::.88 (pktinfo)
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 11: udp ::.88 (pktinfo)
krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 12: udp ::.750 (pktinfo)
Nov 06 12:37:50 master-1 krb5kdc[1081](info): listening on fd 12: udp ::.750 (pktinfo)
Nov 06 12:37:50 master-1 krb5kdc[1081](info): set up 4 sockets
Nov 06 12:37:50 master-1 krb5kdc[1081](info): set up 4 sockets
Nov 06 12:37:50 master-1 krb5kdc[1082](info): commencing operation
Nov 06 12:37:50 master-1 krb5kdc[1082](info): commencing operation
Nov 06 14:29:59 master-1 krb5kdc[1082](debug): Got signal to request exit
Nov 06 14:29:59 master-1 krb5kdc[1082](debug): Got signal to request exit
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 12
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 12
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 11
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 11
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 10
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 10
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 9
Nov 06 14:29:59 master-1 krb5kdc[1082](info): closing down fd 9
Nov 06 14:29:59 master-1 krb5kdc[1082](info): shutting down
Nov 06 14:29:59 master-1 krb5kdc[1082](info): shutting down
Nov 06 14:29:59 master-1 krb5kdc[15418](info): setting up network...
Nov 06 14:29:59 master-1 krb5kdc[15418](info): setting up network...
Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:29:59 master-1 krb5kdc[15418](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:29:59 master-1 krb5kdc[15418](info): set up 2 sockets
Nov 06 14:29:59 master-1 krb5kdc[15418](info): set up 2 sockets
Nov 06 14:29:59 master-1 krb5kdc[15419](info): commencing operation
Nov 06 14:29:59 master-1 krb5kdc[15419](info): commencing operation
Nov 06 14:30:10 master-1 krb5kdc[15419](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573050610, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 14:30:10 master-1 krb5kdc[15419](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573050610, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 14:40:27 master-1 krb5kdc[15419](debug): Got signal to request exit
Nov 06 14:40:27 master-1 krb5kdc[15419](debug): Got signal to request exit
Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 10
Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 10
Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 9
Nov 06 14:40:27 master-1 krb5kdc[15419](info): closing down fd 9
Nov 06 14:40:27 master-1 krb5kdc[15419](info): shutting down
Nov 06 14:40:27 master-1 krb5kdc[15419](info): shutting down
Nov 06 14:40:27 master-1 krb5kdc[16919](info): setting up network...
Nov 06 14:40:27 master-1 krb5kdc[16919](info): setting up network...
Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:40:27 master-1 krb5kdc[16919](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:40:27 master-1 krb5kdc[16919](info): set up 2 sockets
Nov 06 14:40:27 master-1 krb5kdc[16919](info): set up 2 sockets
Nov 06 14:40:27 master-1 krb5kdc[16920](info): commencing operation
Nov 06 14:40:27 master-1 krb5kdc[16920](info): commencing operation
Nov 06 14:44:17 master-1 krb5kdc[16920](debug): Got signal to request exit
Nov 06 14:44:17 master-1 krb5kdc[16920](debug): Got signal to request exit
Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 10
Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 10
Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 9
Nov 06 14:44:17 master-1 krb5kdc[16920](info): closing down fd 9
Nov 06 14:44:17 master-1 krb5kdc[16920](info): shutting down
Nov 06 14:44:17 master-1 krb5kdc[16920](info): shutting down
Nov 06 14:44:17 master-1 krb5kdc[17405](info): setting up network...
Nov 06 14:44:17 master-1 krb5kdc[17405](info): setting up network...
Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:44:17 master-1 krb5kdc[17405](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:44:17 master-1 krb5kdc[17405](info): set up 2 sockets
Nov 06 14:44:17 master-1 krb5kdc[17405](info): set up 2 sockets
Nov 06 14:44:17 master-1 krb5kdc[17406](info): commencing operation
Nov 06 14:44:17 master-1 krb5kdc[17406](info): commencing operation
Nov 06 14:48:42 master-1 krb5kdc[17406](debug): Got signal to request exit
Nov 06 14:48:42 master-1 krb5kdc[17406](debug): Got signal to request exit
Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 10
Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 10
Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 9
Nov 06 14:48:42 master-1 krb5kdc[17406](info): closing down fd 9
Nov 06 14:48:42 master-1 krb5kdc[17406](info): shutting down
Nov 06 14:48:42 master-1 krb5kdc[17406](info): shutting down
Nov 06 14:49:17 master-1 krb5kdc[18050](info): setting up network...
Nov 06 14:49:17 master-1 krb5kdc[18050](info): setting up network...
Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:49:17 master-1 krb5kdc[18050](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:49:17 master-1 krb5kdc[18050](info): set up 2 sockets
Nov 06 14:49:17 master-1 krb5kdc[18050](info): set up 2 sockets
Nov 06 14:49:17 master-1 krb5kdc[18051](info): commencing operation
Nov 06 14:49:17 master-1 krb5kdc[18051](info): commencing operation
Nov 06 14:56:50 master-1 krb5kdc[18051](debug): Got signal to request exit
Nov 06 14:56:50 master-1 krb5kdc[18051](debug): Got signal to request exit
Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 10
Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 10
Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 9
Nov 06 14:56:50 master-1 krb5kdc[18051](info): closing down fd 9
Nov 06 14:56:50 master-1 krb5kdc[18051](info): shutting down
Nov 06 14:56:50 master-1 krb5kdc[18051](info): shutting down
Nov 06 14:56:50 master-1 krb5kdc[18977](info): setting up network...
Nov 06 14:56:50 master-1 krb5kdc[18977](info): setting up network...
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 11: tcp ::.88
Nov 06 14:56:50 master-1 krb5kdc[18977](info): listening on fd 11: tcp ::.88
Nov 06 14:56:50 master-1 krb5kdc[18977](info): set up 4 sockets
Nov 06 14:56:50 master-1 krb5kdc[18977](info): set up 4 sockets
Nov 06 14:56:50 master-1 krb5kdc[18978](info): commencing operation
Nov 06 14:56:50 master-1 krb5kdc[18978](info): commencing operation
Nov 06 15:02:32 master-1 krb5kdc[18978](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573052552, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 15:02:32 master-1 krb5kdc[18978](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573052552, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 15:06:25 master-1 krb5kdc[18978](debug): Got signal to request exit
Nov 06 15:06:25 master-1 krb5kdc[18978](debug): Got signal to request exit
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 11
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 11
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 12
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 12
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 10
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 10
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 9
Nov 06 15:06:25 master-1 krb5kdc[18978](info): closing down fd 9
Nov 06 15:06:25 master-1 krb5kdc[18978](info): shutting down
Nov 06 15:06:25 master-1 krb5kdc[18978](info): shutting down
Nov 06 15:06:25 master-1 krb5kdc[20202](info): setting up network...
Nov 06 15:06:25 master-1 krb5kdc[20202](info): setting up network...
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 11: tcp ::.88
Nov 06 15:06:25 master-1 krb5kdc[20202](info): listening on fd 11: tcp ::.88
Nov 06 15:06:25 master-1 krb5kdc[20202](info): set up 4 sockets
Nov 06 15:06:25 master-1 krb5kdc[20202](info): set up 4 sockets
Nov 06 15:06:25 master-1 krb5kdc[20203](info): commencing operation
Nov 06 15:06:25 master-1 krb5kdc[20203](info): commencing operation
Nov 06 15:09:45 master-1 krb5kdc[20203](debug): Got signal to request exit
Nov 06 15:09:45 master-1 krb5kdc[20203](debug): Got signal to request exit
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 11
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 11
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 12
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 12
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 10
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 10
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 9
Nov 06 15:09:45 master-1 krb5kdc[20203](info): closing down fd 9
Nov 06 15:09:45 master-1 krb5kdc[20203](info): shutting down
Nov 06 15:09:45 master-1 krb5kdc[20203](info): shutting down
Nov 06 15:09:45 master-1 krb5kdc[20588](info): setting up network...
Nov 06 15:09:45 master-1 krb5kdc[20588](info): setting up network...
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 11: tcp ::.88
Nov 06 15:09:45 master-1 krb5kdc[20588](info): listening on fd 11: tcp ::.88
Nov 06 15:09:45 master-1 krb5kdc[20588](info): set up 4 sockets
Nov 06 15:09:45 master-1 krb5kdc[20588](info): set up 4 sockets
Nov 06 15:09:45 master-1 krb5kdc[20589](info): commencing operation
Nov 06 15:09:45 master-1 krb5kdc[20589](info): commencing operation
Nov 06 15:10:23 master-1 krb5kdc[20589](debug): Got signal to request exit
Nov 06 15:10:23 master-1 krb5kdc[20589](debug): Got signal to request exit
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 11
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 11
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 12
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 12
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 10
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 10
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 9
Nov 06 15:10:23 master-1 krb5kdc[20589](info): closing down fd 9
Nov 06 15:10:23 master-1 krb5kdc[20589](info): shutting down
Nov 06 15:10:23 master-1 krb5kdc[20589](info): shutting down
Nov 06 15:10:23 master-1 krb5kdc[20692](info): setting up network...
Nov 06 15:10:23 master-1 krb5kdc[20692](info): setting up network...
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 11: tcp ::.88
Nov 06 15:10:23 master-1 krb5kdc[20692](info): listening on fd 11: tcp ::.88
Nov 06 15:10:23 master-1 krb5kdc[20692](info): set up 4 sockets
Nov 06 15:10:23 master-1 krb5kdc[20692](info): set up 4 sockets
Nov 06 15:10:23 master-1 krb5kdc[20693](info): commencing operation
Nov 06 15:10:23 master-1 krb5kdc[20693](info): commencing operation
Nov 06 15:10:25 master-1 krb5kdc[20693](info): AS_REQ (1 etypes {17}) 10.243.1.196: ISSUE: authtime 1573053025, etypes {rep=17 tkt=17 ses=17}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 15:10:25 master-1 krb5kdc[20693](info): AS_REQ (1 etypes {17}) 10.243.1.196: ISSUE: authtime 1573053025, etypes {rep=17 tkt=17 ses=17}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 15:10:25 master-1 krb5kdc[20693](info): closing down fd 13
Nov 06 15:10:25 master-1 krb5kdc[20693](info): closing down fd 13
Nov 06 15:12:17 master-1 krb5kdc[20693](debug): Got signal to request exit
Nov 06 15:12:17 master-1 krb5kdc[20693](debug): Got signal to request exit
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 11
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 11
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 12
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 12
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 10
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 10
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 9
Nov 06 15:12:17 master-1 krb5kdc[20693](info): closing down fd 9
Nov 06 15:12:17 master-1 krb5kdc[20693](info): shutting down
Nov 06 15:12:17 master-1 krb5kdc[20693](info): shutting down
Nov 06 15:12:17 master-1 krb5kdc[20926](info): setting up network...
Nov 06 15:12:17 master-1 krb5kdc[20926](info): setting up network...
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 11: tcp ::.88
Nov 06 15:12:17 master-1 krb5kdc[20926](info): listening on fd 11: tcp ::.88
Nov 06 15:12:17 master-1 krb5kdc[20926](info): set up 4 sockets
Nov 06 15:12:17 master-1 krb5kdc[20926](info): set up 4 sockets
Nov 06 15:12:17 master-1 krb5kdc[20927](info): commencing operation
Nov 06 15:12:17 master-1 krb5kdc[20927](info): commencing operation
Nov 06 15:12:49 master-1 krb5kdc[20927](debug): Got signal to request exit
Nov 06 15:12:49 master-1 krb5kdc[20927](debug): Got signal to request exit
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 11
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 11
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 12
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 12
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 10
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 10
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 9
Nov 06 15:12:49 master-1 krb5kdc[20927](info): closing down fd 9
Nov 06 15:12:49 master-1 krb5kdc[20927](info): shutting down
Nov 06 15:12:49 master-1 krb5kdc[20927](info): shutting down
Nov 06 15:12:49 master-1 krb5kdc[20993](info): setting up network...
Nov 06 15:12:49 master-1 krb5kdc[20993](info): setting up network...
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 11: tcp ::.88
Nov 06 15:12:49 master-1 krb5kdc[20993](info): listening on fd 11: tcp ::.88
Nov 06 15:12:49 master-1 krb5kdc[20993](info): set up 4 sockets
Nov 06 15:12:49 master-1 krb5kdc[20993](info): set up 4 sockets
Nov 06 15:12:49 master-1 krb5kdc[20994](info): commencing operation
Nov 06 15:12:49 master-1 krb5kdc[20994](info): commencing operation
Nov 06 15:14:48 master-1 krb5kdc[20994](debug): Got signal to request exit
Nov 06 15:14:48 master-1 krb5kdc[20994](debug): Got signal to request exit
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 11
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 11
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 12
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 12
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 10
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 10
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 9
Nov 06 15:14:48 master-1 krb5kdc[20994](info): closing down fd 9
Nov 06 15:14:48 master-1 krb5kdc[20994](info): shutting down
Nov 06 15:14:48 master-1 krb5kdc[20994](info): shutting down
krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm
krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm
krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm
krb5kdc: Configuration file does not specify default realm - while attempting to retrieve default realm
Nov 06 15:16:56 master-1 krb5kdc[21561](info): setting up network...
Nov 06 15:16:56 master-1 krb5kdc[21561](info): setting up network...
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 11: tcp ::.88
Nov 06 15:16:56 master-1 krb5kdc[21561](info): listening on fd 11: tcp ::.88
Nov 06 15:16:56 master-1 krb5kdc[21561](info): set up 4 sockets
Nov 06 15:16:56 master-1 krb5kdc[21561](info): set up 4 sockets
Nov 06 15:16:56 master-1 krb5kdc[21562](info): commencing operation
Nov 06 15:16:56 master-1 krb5kdc[21562](info): commencing operation
Nov 06 17:45:15 master-1 krb5kdc[21562](debug): Got signal to request exit
Nov 06 17:45:15 master-1 krb5kdc[21562](debug): Got signal to request exit
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 11
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 11
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 12
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 12
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 10
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 10
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 9
Nov 06 17:45:15 master-1 krb5kdc[21562](info): closing down fd 9
Nov 06 17:45:15 master-1 krb5kdc[21562](info): shutting down
Nov 06 17:45:15 master-1 krb5kdc[21562](info): shutting down
Nov 06 17:45:15 master-1 krb5kdc[8642](info): setting up network...
Nov 06 17:45:15 master-1 krb5kdc[8642](info): setting up network...
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 12: tcp 0.0.0.0.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): listening on fd 11: tcp ::.88
Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets
Nov 06 17:45:15 master-1 krb5kdc[8642](info): set up 4 sockets
Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation
Nov 06 17:45:15 master-1 krb5kdc[8643](info): commencing operation
Nov 06 17:58:08 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573063088, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 06 17:58:08 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573063088, etypes {rep=18 tkt=18 ses=18}, cloudera-scm/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 07 09:03:31 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573117411, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
Nov 07 09:03:31 master-1 krb5kdc[8643](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573117411, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
Nov 07 09:03:51 master-1 krb5kdc[8643](Error): TCP client 10.243.1.196.60500 wants 2147484324 bytes, cap is 1048572
Nov 07 09:03:51 master-1 krb5kdc[8643](Error): TCP client 10.243.1.196.60500 wants 2147484324 bytes, cap is 1048572
Nov 07 09:03:51 master-1 krb5kdc[8643](info): closing down fd 13
Nov 07 09:03:51 master-1 krb5kdc[8643](info): closing down fd 13
Nov 07 09:05:36 master-1 krb5kdc[8643](debug): Got signal to request exit
Nov 07 09:05:36 master-1 krb5kdc[8643](debug): Got signal to request exit
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 11
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 11
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 12
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 12
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 10
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 10
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 9
Nov 07 09:05:36 master-1 krb5kdc[8643](info): closing down fd 9
Nov 07 09:05:36 master-1 krb5kdc[8643](info): shutting down
Nov 07 09:05:36 master-1 krb5kdc[8643](info): shutting down
Nov 07 09:05:36 master-1 krb5kdc[29966](info): setting up network...
Nov 07 09:05:36 master-1 krb5kdc[29966](info): setting up network...
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 12: tcp 0.0.0.0.88
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 12: tcp 0.0.0.0.88
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 11: tcp ::.88
Nov 07 09:05:36 master-1 krb5kdc[29966](info): listening on fd 11: tcp ::.88
Nov 07 09:05:36 master-1 krb5kdc[29966](info): set up 4 sockets
Nov 07 09:05:36 master-1 krb5kdc[29966](info): set up 4 sockets
Nov 07 09:05:36 master-1 krb5kdc[29967](info): commencing operation
Nov 07 09:05:36 master-1 krb5kdc[29967](info): commencing operation
Nov 07 09:10:10 master-1 krb5kdc[29967](debug): Got signal to request exit
Nov 07 09:10:10 master-1 krb5kdc[29967](debug): Got signal to request exit
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 11
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 11
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 12
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 12
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 10
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 10
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 9
Nov 07 09:10:10 master-1 krb5kdc[29967](info): closing down fd 9
Nov 07 09:10:10 master-1 krb5kdc[29967](info): shutting down
Nov 07 09:10:10 master-1 krb5kdc[29967](info): shutting down
Nov 07 09:11:13 master-1 krb5kdc[1093](info): setting up network...
Nov 07 09:11:13 master-1 krb5kdc[1093](info): setting up network...
Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 09:11:13 master-1 krb5kdc[1093](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: Address already in use - Cannot bind server socket on ::.88
krb5kdc: Address already in use - Cannot bind server socket on ::.88
Nov 07 09:11:13 master-1 krb5kdc[1093](info): set up 2 sockets
Nov 07 09:11:13 master-1 krb5kdc[1093](info): set up 2 sockets
Nov 07 09:11:13 master-1 krb5kdc[1130](info): commencing operation
Nov 07 09:11:13 master-1 krb5kdc[1130](info): commencing operation
Nov 07 09:14:09 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118049, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
Nov 07 09:14:09 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118049, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
Nov 07 09:16:38 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118198, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 07 09:16:38 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573118198, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 07 09:34:53 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119293, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
Nov 07 09:34:53 master-1 krb5kdc[1130](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119293, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for kadmin/master-1.dev.edl.gcp.domain.it@EDHDEV.COM
Nov 07 09:37:40 master-1 krb5kdc[1130](debug): Got signal to request exit
Nov 07 09:37:40 master-1 krb5kdc[1130](debug): Got signal to request exit
Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 10
Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 10
Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 9
Nov 07 09:37:40 master-1 krb5kdc[1130](info): closing down fd 9
Nov 07 09:37:40 master-1 krb5kdc[1130](info): shutting down
Nov 07 09:37:40 master-1 krb5kdc[1130](info): shutting down
Nov 07 09:37:40 master-1 krb5kdc[8394](info): setting up network...
Nov 07 09:37:40 master-1 krb5kdc[8394](info): setting up network...
Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 09:37:40 master-1 krb5kdc[8394](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: Address already in use - Cannot bind server socket on ::.88
krb5kdc: Address already in use - Cannot bind server socket on ::.88
Nov 07 09:37:40 master-1 krb5kdc[8394](info): set up 2 sockets
Nov 07 09:37:40 master-1 krb5kdc[8394](info): set up 2 sockets
Nov 07 09:37:40 master-1 krb5kdc[8396](info): commencing operation
Nov 07 09:37:40 master-1 krb5kdc[8396](info): commencing operation
Nov 07 09:37:55 master-1 krb5kdc[8396](debug): Got signal to request exit
Nov 07 09:37:55 master-1 krb5kdc[8396](debug): Got signal to request exit
Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 10
Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 10
Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 9
Nov 07 09:37:55 master-1 krb5kdc[8396](info): closing down fd 9
Nov 07 09:37:55 master-1 krb5kdc[8396](info): shutting down
Nov 07 09:37:55 master-1 krb5kdc[8396](info): shutting down
Nov 07 09:38:00 master-1 krb5kdc[8468](info): setting up network...
Nov 07 09:38:00 master-1 krb5kdc[8468](info): setting up network...
Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 09:38:00 master-1 krb5kdc[8468](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: Address already in use - Cannot bind server socket on ::.88
krb5kdc: Address already in use - Cannot bind server socket on ::.88
Nov 07 09:38:00 master-1 krb5kdc[8468](info): set up 2 sockets
Nov 07 09:38:00 master-1 krb5kdc[8468](info): set up 2 sockets
Nov 07 09:38:00 master-1 krb5kdc[8469](info): commencing operation
Nov 07 09:38:00 master-1 krb5kdc[8469](info): commencing operation
Nov 07 09:38:08 master-1 krb5kdc[8469](debug): Got signal to request exit
Nov 07 09:38:08 master-1 krb5kdc[8469](debug): Got signal to request exit
Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 10
Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 10
Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 9
Nov 07 09:38:08 master-1 krb5kdc[8469](info): closing down fd 9
Nov 07 09:38:08 master-1 krb5kdc[8469](info): shutting down
Nov 07 09:38:08 master-1 krb5kdc[8469](info): shutting down
Nov 07 09:42:37 master-1 krb5kdc[9029](info): setting up network...
Nov 07 09:42:37 master-1 krb5kdc[9029](info): setting up network...
Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 09:42:37 master-1 krb5kdc[9029](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: Address already in use - Cannot bind server socket on ::.88
krb5kdc: Address already in use - Cannot bind server socket on ::.88
Nov 07 09:42:37 master-1 krb5kdc[9029](info): set up 2 sockets
Nov 07 09:42:37 master-1 krb5kdc[9029](info): set up 2 sockets
Nov 07 09:42:37 master-1 krb5kdc[9030](info): commencing operation
Nov 07 09:42:37 master-1 krb5kdc[9030](info): commencing operation
Nov 07 09:42:55 master-1 krb5kdc[9030](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119775, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 07 09:42:55 master-1 krb5kdc[9030](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573119775, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 07 12:32:38 master-1 krb5kdc[9030](debug): Got signal to request exit
Nov 07 12:32:38 master-1 krb5kdc[9030](debug): Got signal to request exit
Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 10
Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 10
Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 9
Nov 07 12:32:38 master-1 krb5kdc[9030](info): closing down fd 9
Nov 07 12:32:38 master-1 krb5kdc[9030](info): shutting down
Nov 07 12:32:38 master-1 krb5kdc[9030](info): shutting down
Nov 07 12:32:42 master-1 krb5kdc[30870](info): setting up network...
Nov 07 12:32:42 master-1 krb5kdc[30870](info): setting up network...
Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 12:32:42 master-1 krb5kdc[30870](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: Address already in use - Cannot bind server socket on ::.88
krb5kdc: Address already in use - Cannot bind server socket on ::.88
Nov 07 12:32:42 master-1 krb5kdc[30870](info): set up 2 sockets
Nov 07 12:32:42 master-1 krb5kdc[30870](info): set up 2 sockets
Nov 07 12:32:42 master-1 krb5kdc[30871](info): commencing operation
Nov 07 12:32:42 master-1 krb5kdc[30871](info): commencing operation
Nov 07 12:40:14 master-1 krb5kdc[30871](debug): Got signal to request exit
Nov 07 12:40:14 master-1 krb5kdc[30871](debug): Got signal to request exit
Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 10
Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 10
Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 9
Nov 07 12:40:14 master-1 krb5kdc[30871](info): closing down fd 9
Nov 07 12:40:14 master-1 krb5kdc[30871](info): shutting down
Nov 07 12:40:14 master-1 krb5kdc[30871](info): shutting down
Nov 07 12:40:19 master-1 krb5kdc[31877](info): setting up network...
Nov 07 12:40:19 master-1 krb5kdc[31877](info): setting up network...
Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 9: udp 0.0.0.0.88 (pktinfo)
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo)
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked
krb5kdc: Address already in use - Cannot bind server socket on ::.88
krb5kdc: Address already in use - Cannot bind server socket on ::.88
Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets
Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets
Nov 07 12:40:19 master-1 krb5kdc[31878](info): commencing operation
Nov 07 12:40:19 master-1 krb5kdc[31878](info): commencing operation
Nov 07 13:05:51 master-1 krb5kdc[31878](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573131951, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM
Nov 07 13:05:51 master-1 krb5kdc[31878](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.243.1.196: ISSUE: authtime 1573131951, etypes {rep=18 tkt=18 ses=18}, admin/admin@EDHDEV.COM for krbtgt/EDHDEV.COM@EDHDEV.COM

Thanks

Shelton · ‎11-07-2019

@svasi

For the logs I can see that the UDP port 88 used by the KDC is already in use by some application.

Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo)
Nov 07 12:40:19 master-1 krb5kdc[31877](info): listening on fd 10: udp ::.88 (pktinfo) krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: setsockopt(11,IPV6_V6ONLY,1) worked krb5kdc: Address already in use - Cannot bind server socket on ::.88 krb5kdc:
Address already in use - Cannot bind server socket on ::.88
Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets
Nov 07 12:40:19 master-1 krb5kdc[31877](info): set up 2 sockets

The default ports used by Kerberos are port 88 for the KDC1 and port 749 for the admin server. You can, however, choose to run on other ports, as long as they are specified in each host's /etc/services and krb5.conf files, and the kdc.conf file on each KDC

Solution one

Check for the PID of port 88
# netstat -tupln

kill PID
# kill -9 <PID>

Retry your kerberiszation

Solution 2

Change the port to something like 8888 in /etc/services and krb5.conf files, and the kdc.conf

Restart the kdc and kadmin and retry the kerberization

Cloudera Community

Support Questions

Enable Kerberos - Setup KDC Account - Continue enabled but not working