HMaster server kerberos debug flag enabled logs: ----------------------------------------------- Native config name: /etc/krb5.conf Loaded from native config >>>KinitOptions cache name is /tmp/krb5cc_1443726158 >>>DEBUG: TCPClient reading 207 bytes >>> KrbKdcReq send: #bytes read=207 >>>Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 23, salt = >>>Pre-Authentication Data: PA-DATA type = 19 PA-ETYPE-INFO2 etype = 23, salt = null, s2kparams = null >>>Pre-Authentication Data: PA-DATA type = 2 PA-ENC-TIMESTAMP >>>Pre-Authentication Data: PA-DATA type = 16 >>>Pre-Authentication Data: PA-DATA type = 15 >>> KDCRep: init() encoding tag is 126 req type is 11 >>>KRBError: sTime is Fri Jan 20 18:17:02 NZDT 2017 1484889422000 suSec is 752029 error code is 25 error Message is Additional pre-authentication required realm is ADC.EXAMPLE.COM sname is krbtgt/ADC.EXAMPLE.COM eData provided. msgType is 30 >>>Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 23, salt = Ordering keys wrt default_tkt_enctypes list default etypes for default_tkt_enctypes: 23. Debug is true storeKey false useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is /data/yarn/nm/usercache/srvuser/appcache/application_1484884580211_0007/container_1484884580211_0007_01_000002/keytabs/hbase.keytab refreshKrb5Config is false principal is srvuser/a1.example.com@ADC.EXAMPLE.COM tryFirstPass is false useFirstPass is false storePass is false clearPass is false Added key: 23version: 1 >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType >>> KrbAsRep cons in KrbAsReq.getReply srvuser/a1.example.com principal is KrbAsReq.getReply srvuser/a1.example.com@ADC.EXAMPLE.COM Will use keytab Commit Succeeded Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:03 NZDT 2017 Entered Krb5Context.initSecContext with state=STATE_NEW Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:03 NZDT 2017 Service ticket not found in the subject >>> Credentials acquireServiceCreds: same realm default etypes for default_tgs_enctypes: 23. >>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType >>>DEBUG: TCPClient reading 1898 bytes >>> KrbKdcReq send: #bytes read=1898 >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType >>> KrbApReq: APOptions are 00000000 00000000 00000000 00000000 >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType Krb5Context setting mySeqNumber to: 267126047 Krb5Context setting peerSeqNumber to: 0 Created InitSecContextToken: Krb5Context.unwrap: token=[60 30 06 09 2a 86 48 86 f7 12 01 02 02 02 01 11 00 ff ff ff ff 29 e6 ae 2e 64 64 e8 75 94 18 88 79 a2 78 5f d9 41 76 a6 73 8d 44 68 0f 01 01 00 00 01 ] Krb5Context.unwrap: data=[01 01 00 00 ] Krb5Context.wrap: data=[01 01 00 00 73 72 76 64 65 76 61 6e 61 6c 79 74 69 63 73 69 6e 66 61 2f 64 78 6c 68 75 62 30 32 2e 6e 7a 2e 74 68 65 6e 61 74 69 6f 6e 61 6c 2e 63 6f 6d 40 42 4e 5a 4e 41 47 2e 4e 5a 2e 54 48 45 4e 41 54 49 4f 4e 41 4c 2e 43 4f 4d ] Krb5Context.wrap: token=[60 79 06 09 2a 86 48 86 f7 12 01 02 02 02 01 11 00 ff ff ff ff f9 d7 94 f3 c8 4f 3c 87 9f 4a 04 a7 00 dd 38 28 89 57 86 79 7c 84 bd 47 01 01 00 00 73 72 76 64 65 76 61 6e 61 6c 79 74 69 63 73 69 6e 66 61 2f 64 78 6c 68 75 62 30 32 2e 6e 7a 2e 74 68 65 6e 61 74 69 6f 6e 61 6c 2e 63 6f 6d 40 42 4e 5a 4e 41 47 2e 4e 5a 2e 54 48 45 4e 41 54 49 4f 4e 41 4c 2e 43 4f 4d 01 ] Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:03 NZDT 2017 Entered Krb5Context.initSecContext with state=STATE_NEW Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:03 NZDT 2017 Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to zookeeper/a2.example.com@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:03 NZDT 2017 Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:02 NZDT 2017 Entered Krb5Context.initSecContext with state=STATE_NEW Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:02 NZDT 2017 Service ticket not found in the subject >>> Credentials acquireServiceCreds: same realm default etypes for default_tgs_enctypes: 23. >>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType Client Principal = srvuser/a1.example.com@ADC.EXAMPLE.COM Server Principal = zookeeper/a2.example.com@ADC.EXAMPLE.COM Session Key = EncryptionKey: keyType=23 keyBytes (hex dump)= 0000: 8D 62 C2 4E 19 37 5C 40 63 EF 04 91 40 D4 7F F4 .b.N.7\@c...@... Forwardable Ticket true Forwarded Ticket false Proxiable Ticket false Proxy Ticket false Postdated Ticket false Renewable Ticket false Initial Ticket false Auth Time = Fri Jan 20 18:17:03 NZDT 2017 Start Time = Fri Jan 20 18:17:03 NZDT 2017 End Time = Sat Jan 21 04:17:03 NZDT 2017 Renew Till = null Client Addresses Null Krb5Context.unwrap: token=[60 30 06 09 2a 86 48 86 f7 12 01 02 02 02 01 11 00 ff ff ff ff 0e 6d 9a a1 cb 58 f7 71 09 79 5f 92 71 3d b6 5b e0 35 be 28 22 53 66 0b 01 01 00 00 01 ] Krb5Context.unwrap: data=[01 01 00 00 ] Krb5Context.wrap: data=[01 01 00 00 73 72 76 64 65 76 61 6e 61 6c 79 74 69 63 73 69 6e 66 61 2f 64 78 6c 68 75 62 30 32 2e 6e 7a 2e 74 68 65 6e 61 74 69 6f 6e 61 6c 2e 63 6f 6d 40 42 4e 5a 4e 41 47 2e 4e 5a 2e 54 48 45 4e 41 54 49 4f 4e 41 4c 2e 43 4f 4d ] Krb5Context.wrap: token=[60 79 06 09 2a 86 48 86 f7 12 01 02 02 02 01 11 00 ff ff ff ff 1c 02 3f 22 4a 7f 93 82 59 cd 44 3a 57 27 9d 3c 30 88 59 3d 9b b2 20 2d 01 01 00 00 73 72 76 64 65 76 61 6e 61 6c 79 74 69 63 73 69 6e 66 61 2f 64 78 6c 68 75 62 30 32 2e 6e 7a 2e 74 68 65 6e 61 74 69 6f 6e 61 6c 2e 63 6f 6d 40 42 4e 5a 4e 41 47 2e 4e 5a 2e 54 48 45 4e 41 54 49 4f 4e 41 4c 2e 43 4f 4d 01 ] Found KeyTab Found KerberosKey for srvuser/a1.example.com@ADC.EXAMPLE.COM Entered Krb5Context.acceptSecContext with state=STATE_NEW Added key: 23version: 1 Ordering keys wrt default_tkt_enctypes list default etypes for default_tkt_enctypes: 23. Found KerberosKey for srvuser/a1.example.com@ADC.EXAMPLE.COM Entered Krb5Context.acceptSecContext with state=STATE_NEW Added key: 23version: 1 Ordering keys wrt default_tkt_enctypes list default etypes for default_tkt_enctypes: 23. Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:02 NZDT 2017 Entered Krb5Context.initSecContext with state=STATE_NEW Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to krbtgt/ADC.EXAMPLE.COM@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:02 NZDT 2017 Found ticket for srvuser/a1.example.com@ADC.EXAMPLE.COM to go to hdfs/a2.example.com@ADC.EXAMPLE.COM expiring on Sat Jan 21 04:17:02 NZDT 2017 Found service ticket in the subjectTicket (hex) = Client Principal = srvuser/a1.example.com@ADC.EXAMPLE.COM Server Principal = hdfs/a2.example.com@ADC.EXAMPLE.COM Session Key = EncryptionKey: keyType=23 keyBytes (hex dump)= 0000: 67 EE 48 26 3A DA FD 17 C4 BF 4B 94 E8 FA F5 4D g.H&:.....K....M Forwardable Ticket true Forwarded Ticket false Proxiable Ticket false Proxy Ticket false Postdated Ticket false Renewable Ticket false Initial Ticket false Auth Time = Fri Jan 20 18:17:02 NZDT 2017 Start Time = Fri Jan 20 18:17:04 NZDT 2017 End Time = Sat Jan 21 04:17:02 NZDT 2017 Renew Till = null Client Addresses Null >>> KrbApReq: APOptions are 00100000 00000000 00000000 00000000 >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType Krb5Context setting mySeqNumber to: 618269098 Created InitSecContextToken: Entered Krb5Context.initSecContext with state=STATE_IN_PROCESS >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType Krb5Context setting peerSeqNumber to: 796327394 Krb5Context.unwrap: token=[60 30 06 09 2a 86 48 86 f7 12 01 02 02 02 01 11 00 ff ff ff ff 96 4c 1e 1e 77 74 a7 8d da f3 b0 28 49 c4 83 bf 29 a4 25 85 58 bb d1 24 01 01 00 00 01 ] Krb5Context.unwrap: data=[01 01 00 00 ] Krb5Context.wrap: data=[01 01 00 00 ] Krb5Context.wrap: token=[60 30 06 09 2a 86 48 86 f7 12 01 02 02 02 01 11 00 ff ff ff ff 38 14 82 e3 e2 a6 a9 65 6e 7f 0e 55 90 14 c4 0c 23 87 c5 2a 8e 81 10 99 01 01 00 00 01 ] Found KeyTab Found KerberosKey for srvuser/a1.example.com@ADC.EXAMPLE.COM Entered Krb5Context.acceptSecContext with state=STATE_NEW Added key: 23version: 1 Ordering keys wrt default_tkt_enctypes list default etypes for default_tkt_enctypes: 23.