Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hbase client error when using kerberos

hbase client error when using kerberos

New Contributor

I have encounter following error:

 

Config name: C:\Windows\krb5.ini
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:host.name=9.78.211.112
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_33
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:java.home=C:\Java\jdk1.6\jre
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:java.class.path=C:\project\github\Big_data_example\me.huiwen.example.hbase.kerberos\target\classes;C:\Users\sfb_ADMIN\.m2\repository\junit\junit\4.11\junit-4.11.jar;C:\Users\sfb_ADMIN\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Java\jdk1.7.0_21\lib\tools.jar;C:\Users\sfb_ADMIN\.m2\repository\log4j\log4j\1.2.15\log4j-1.2.15.jar;C:\Users\sfb_ADMIN\.m2\repository\org\apache\hbase\hbase-client\0.95.0\hbase-client-0.95.0.jar;C:\Users\sfb_ADMIN\.m2\repository\org\apache\hbase\hbase-common\0.95.0\hbase-common-0.95.0.jar;C:\Users\sfb_ADMIN\.m2\repository\com\google\guava\guava\12.0.1\guava-12.0.1.jar;C:\Users\sfb_ADMIN\.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-logging\commons-logging\1.1.1\commons-logging-1.1.1.jar;C:\Users\sfb_ADMIN\.m2\repository\org\slf4j\slf4j-api\1.4.3\slf4j-api-1.4.3.jar;C:\Users\sfb_ADMIN\.m2\repository\org\slf4j\slf4j-log4j12\1.4.3\slf4j-log4j12-1.4.3.jar;C:\Users\sfb_ADMIN\.m2\repository\org\apache\hbase\hbase-protocol\0.95.0\hbase-protocol-0.95.0.jar;C:\Users\sfb_ADMIN\.m2\repository\com\google\protobuf\protobuf-java\2.4.1\protobuf-java-2.4.1.jar;C:\Users\sfb_ADMIN\.m2\repository\org\apache\zookeeper\zookeeper\3.4.5\zookeeper-3.4.5.jar;C:\Users\sfb_ADMIN\.m2\repository\org\cloudera\htrace\htrace\1.50\htrace-1.50.jar;C:\Users\sfb_ADMIN\.m2\repository\org\mortbay\jetty\jetty-util\6.1.26\jetty-util-6.1.26.jar;C:\Users\sfb_ADMIN\.m2\repository\io\netty\netty\3.5.9.Final\netty-3.5.9.Final.jar;C:\Users\sfb_ADMIN\.m2\repository\org\apache\hadoop\hadoop-core\1.1.2\hadoop-core-1.1.2.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-cli\commons-cli\1.2\commons-cli-1.2.jar;C:\Users\sfb_ADMIN\.m2\repository\xmlenc\xmlenc\0.52\xmlenc-0.52.jar;C:\Users\sfb_ADMIN\.m2\repository\com\sun\jersey\jersey-core\1.8\jersey-core-1.8.jar;C:\Users\sfb_ADMIN\.m2\repository\com\sun\jersey\jersey-json\1.8\jersey-json-1.8.jar;C:\Users\sfb_ADMIN\.m2\repository\org\codehaus\jettison\jettison\1.1\jettison-1.1.jar;C:\Users\sfb_ADMIN\.m2\repository\stax\stax-api\1.0.1\stax-api-1.0.1.jar;C:\Users\sfb_ADMIN\.m2\repository\com\sun\xml\bind\jaxb-impl\2.2.3-1\jaxb-impl-2.2.3-1.jar;C:\Users\sfb_ADMIN\.m2\repository\javax\xml\bind\jaxb-api\2.2.2\jaxb-api-2.2.2.jar;C:\Users\sfb_ADMIN\.m2\repository\javax\xml\stream\stax-api\1.0-2\stax-api-1.0-2.jar;C:\Users\sfb_ADMIN\.m2\repository\javax\activation\activation\1.1\activation-1.1.jar;C:\Users\sfb_ADMIN\.m2\repository\org\codehaus\jackson\jackson-xc\1.7.1\jackson-xc-1.7.1.jar;C:\Users\sfb_ADMIN\.m2\repository\com\sun\jersey\jersey-server\1.8\jersey-server-1.8.jar;C:\Users\sfb_ADMIN\.m2\repository\asm\asm\3.1\asm-3.1.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-io\commons-io\2.1\commons-io-2.1.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-httpclient\commons-httpclient\3.0.1\commons-httpclient-3.0.1.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-codec\commons-codec\1.4\commons-codec-1.4.jar;C:\Users\sfb_ADMIN\.m2\repository\org\apache\commons\commons-math\2.1\commons-math-2.1.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-configuration\commons-configuration\1.6\commons-configuration-1.6.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-collections\commons-collections\3.2.1\commons-collections-3.2.1.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-lang\commons-lang\2.4\commons-lang-2.4.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-digester\commons-digester\1.8\commons-digester-1.8.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-beanutils\commons-beanutils\1.7.0\commons-beanutils-1.7.0.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-beanutils\commons-beanutils-core\1.8.0\commons-beanutils-core-1.8.0.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-net\commons-net\1.4.1\commons-net-1.4.1.jar;C:\Users\sfb_ADMIN\.m2\repository\org\mortbay\jetty\jetty\6.1.26\jetty-6.1.26.jar;C:\Users\sfb_ADMIN\.m2\repository\org\mortbay\jetty\servlet-api\2.5-20081211\servlet-api-2.5-20081211.jar;C:\Users\sfb_ADMIN\.m2\repository\tomcat\jasper-runtime\5.5.12\jasper-runtime-5.5.12.jar;C:\Users\sfb_ADMIN\.m2\repository\tomcat\jasper-compiler\5.5.12\jasper-compiler-5.5.12.jar;C:\Users\sfb_ADMIN\.m2\repository\org\mortbay\jetty\jsp-api-2.1\6.1.14\jsp-api-2.1-6.1.14.jar;C:\Users\sfb_ADMIN\.m2\repository\org\mortbay\jetty\servlet-api-2.5\6.1.14\servlet-api-2.5-6.1.14.jar;C:\Users\sfb_ADMIN\.m2\repository\org\mortbay\jetty\jsp-2.1\6.1.14\jsp-2.1-6.1.14.jar;C:\Users\sfb_ADMIN\.m2\repository\ant\ant\1.6.5\ant-1.6.5.jar;C:\Users\sfb_ADMIN\.m2\repository\commons-el\commons-el\1.0\commons-el-1.0.jar;C:\Users\sfb_ADMIN\.m2\repository\com\github\stephenc\findbugs\findbugs-annotations\1.3.9-1\findbugs-annotations-1.3.9-1.jar
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:java.library.path=C:\Java\jdk1.6\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:/Java/jdk1.7.0_21/bin/../jre/bin/server;C:/Java/jdk1.7.0_21/bin/../jre/bin;C:/Java/jdk1.7.0_21/bin/../jre/lib/amd64;C:\develop\Ruby193\bin;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\Quest Software\PuTTY\;C:\develop\apache-maven-3.0.4\bin;C:\Java\jdk1.7.0_21\bin;C:\develop\mingw64\bin;C:\wsa\deepqa\blade\latest_build\lib\x86_64\xsg;C:\develop\mingw64\x86_64-w64-mingw32\lib;C:\develop\mingw64\lib;C:\develop\mingw64\x86_64-w64-mingw32\lib;C:\develop\play-2.0.3;C:\Program Files\Mercurial;c:\notes;C:\app\Git\bin;C:\develop\Python27;C:\develop\Python27\Scripts;C:\develop\eclipse-jee-galileo-SR2-win32\eclipse;;.
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=C:\Users\sfb_AD~1\AppData\Local\Temp\
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:os.name=Windows 7
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:os.version=6.1
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:user.name=henry
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:user.home=C:\Users\sfb_ADMIN
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Client environment:user.dir=C:\project\github\Big_data_example\me.huiwen.example.hbase.kerberos
13/08/09 15:53:18 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=dpev209.innovate.sfb.com:2181 sessionTimeout=600000 watcher=hconnection-0x42552c
13/08/09 15:53:18 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is hconnection-0x42552c
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 63; type: 18
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 47; type: 17
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 55; type: 16
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 47; type: 23
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 39; type: 8
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 39; type: 3
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 63; type: 18
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 47; type: 17
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 55; type: 16
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 47; type: 23
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 39; type: 8
>>> KeyTabInputStream, readName(): sfb.COM
>>> KeyTabInputStream, readName(): zkcli
>>> KeyTab: load() entry length: 39; type: 3
Added key: 3version: 8
Found unsupported keytype (8) for zkcli@sfb.COM
Added key: 23version: 8
Added key: 16version: 8
Added key: 17version: 8
Added key: 18version: 8
Added key: 3version: 7
Found unsupported keytype (8) for zkcli@sfb.COM
Added key: 23version: 7
Added key: 16version: 7
Added key: 17version: 7
Added key: 18version: 7
Ordering keys wrt default_tkt_enctypes list
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 3 1 23 16 17 18.
>>> KdcAccessibility: reset
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 3 1 23 16 17 18.
>>> KrbAsReq calling createMessage
>>> KrbAsReq in createMessage
>>> KrbKdcReq send: kdc=dpev210.innovate.sfb.com UDP:88, timeout=30000, number of retries =3, #bytes=139
>>> KDCCommunication: kdc=dpev210.innovate.sfb.com UDP:88, timeout=30000,Attempt =1, #bytes=139
>>> KrbKdcReq send: #bytes read=495
>>> KrbKdcReq send: #bytes read=495
>>> KdcAccessibility: remove dpev210.innovate.sfb.com
>>> EType: sun.security.krb5.internal.crypto.DesCbcMd5EType
>>> KrbAsRep cons in KrbAsReq.getReply zkcli
13/08/09 15:53:19 INFO zookeeper.Login: successfully logged in.
13/08/09 15:53:19 INFO zookeeper.Login: TGT refresh thread started.
13/08/09 15:53:19 INFO client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
13/08/09 15:53:19 INFO zookeeper.Login: TGT valid starting at: Fri Aug 09 15:53:39 GMT 2013
13/08/09 15:53:19 INFO zookeeper.Login: TGT expires: Sat Aug 10 03:53:39 GMT 2013
13/08/09 15:53:19 INFO zookeeper.Login: TGT refresh sleeping until: Sat Aug 10 01:29:42 GMT 2013
13/08/09 15:53:19 INFO zookeeper.ClientCnxn: Opening socket connection to server dpev209.innovate.sfb.com/9.45.92.37:2181. Will attempt to SASL-authenticate using Login Context section 'Client'
13/08/09 15:53:20 INFO zookeeper.ClientCnxn: Socket connection established to dpev209.innovate.sfb.com/9.45.92.37:2181, initiating session
13/08/09 15:53:20 INFO zookeeper.ClientCnxn: Session establishment complete on server dpev209.innovate.sfb.com/9.45.92.37:2181, sessionid = 0x1406363f3110107, negotiated timeout = 600000
Found ticket for zkcli@sfb.COM to go to krbtgt/sfb.COM@sfb.COM expiring on Sat Aug 10 03:53:39 GMT 2013
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for zkcli@sfb.COM to go to krbtgt/sfb.COM@sfb.COM expiring on Sat Aug 10 03:53:39 GMT 2013
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 3 1 23 16 17 18.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.DesCbcCrcEType
>>>crc32: c65aeb1
>>>crc32: 1100011001011010111010110001
>>> KrbKdcReq send: kdc=dpev210.innovate.sfb.com UDP:88, timeout=30000, number of retries =3, #bytes=539
>>> KDCCommunication: kdc=dpev210.innovate.sfb.com UDP:88, timeout=30000,Attempt =1, #bytes=539
>>> KrbKdcReq send: #bytes read=538
>>> KrbKdcReq send: #bytes read=538
>>> KdcAccessibility: remove dpev210.innovate.sfb.com
>>> EType: sun.security.krb5.internal.crypto.DesCbcCrcEType
>>>crc32: 1963d0dd
>>>crc32: 11001011000111101000011011101
>>> KrbApReq: APOptions are 00000000 00000000 00000000 00000000
>>> EType: sun.security.krb5.internal.crypto.DesCbcCrcEType
>>>crc32: 5812045a
>>>crc32: 1011000000100100000010001011010
Krb5Context setting mySeqNumber to: 137426464
Krb5Context setting peerSeqNumber to: 0
Created InitSecContextToken:
0000: 01 00 6E 82 01 C8 30 82 01 C4 A0 03 02 01 05 A1 ..n...0.........
0010: 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 82 01 ................
0020: 04 61 82 01 00 30 81 FD A0 03 02 01 05 A1 09 1B .a...0..........
0030: 07 49 42 4D 2E 43 4F 4D A2 30 30 2E A0 03 02 01 .sfb.COM.00.....
0040: 00 A1 27 30 25 1B 09 7A 6F 6F 6B 65 65 70 65 72 ..'0%..zookeeper
0050: 1B 18 64 70 65 76 32 30 39 2E 69 6E 6E 6F 76 61 ..dpev209.innova
0060: 74 65 2E 69 62 6D 2E 63 6F 6D A3 81 B8 30 81 B5 te.sfb.com...0..
0070: A0 03 02 01 12 A1 03 02 01 04 A2 81 A8 04 81 A5 ................
0080: E8 B6 34 EB 89 EB 15 54 2A 80 BF BA FB FD 6A 5A ..4....T*.....jZ
0090: F7 2F CE A9 D2 36 FB B8 A0 AA 15 09 C8 37 50 15 ./...6.......7P.
00A0: 37 6E A4 93 D6 B6 08 BD 0A C2 46 84 F4 BC 7C AF 7n........F.....
00B0: F4 42 73 7B AA E4 9C A2 0E F6 E9 EA 67 CE A0 7D .Bs.........g...
00C0: D9 81 D9 F0 56 1B 70 FA 4F D1 92 EA 0D 13 59 A0 ....V.p.O.....Y.
00D0: 20 9D AA EF 18 77 F1 56 F4 7A DA 5E 8D CC 26 AC ....w.V.z.^..&.
00E0: 30 C0 E4 67 F6 65 2A F5 D1 EC 98 F2 6C 7E 61 2D 0..g.e*.....l.a-
00F0: 1A E5 C6 50 D1 35 21 50 95 3C 76 2F 08 57 30 FB ...P.5!P.<v/.W0.
0100: 77 84 F1 BF 8C DA EA AA 61 CE 85 FA 11 36 43 B8 w.......a....6C.
0110: 8D BF 56 06 D0 E9 D6 BD F6 7A DC A1 2A D6 80 F8 ..V......z..*...
0120: A9 47 1A 3F 2E A4 81 A6 30 81 A3 A0 03 02 01 01 .G.?....0.......
0130: A2 81 9B 04 81 98 96 08 25 CA 86 98 86 5E 1B 47 ........%....^.G
0140: 88 FB 57 FC 25 63 F7 91 8E 37 A2 5A 27 AF B0 C8 ..W.%c...7.Z'...
0150: E1 C3 F6 BE B5 87 D7 43 57 60 7D 92 19 A6 85 3B .......CW`.....;
0160: 08 6C 01 9F 24 C7 EA E4 16 D1 1A B7 53 85 F8 56 .l..$.......S..V
0170: 52 A0 1A 24 C4 3C EA 9B 29 15 0A E1 54 1E 6C 37 R..$.<..)...T.l7
0180: A0 A9 BA 73 62 37 3B 55 24 42 DE D4 CF 6E 5F F9 ...sb7;U$B...n_.
0190: 7E E9 65 D2 D4 76 9C DE 1B 27 21 41 C2 5A D9 A4 ..e..v...'!A.Z..
01A0: 12 48 62 55 22 B0 D1 59 1B B7 6E CC DD 2B 61 7E .HbU"..Y..n..+a.
01B0: 7B FC CA 4C E6 C2 99 23 63 BC 4A 21 1D E6 C8 91 ...L...#c.J!....
01C0: FB 4E 20 74 37 C4 42 F6 4F 3E FC 7B F8 76 .N t7.B.O>...v

Krb5Context.unwrap: token=[60 33 06 09 2a 86 48 86 f7 12 01 02 02 02 01 00 00 ff ff ff ff 36 35 72 7c 19 9e 17 bc d7 17 c4 1f 9c f7 e5 c4 95 a6 97 2b 8e ae f0 e7 01 01 00 00 04 04 04 04 ]
Krb5Context.unwrap: data=[01 01 00 00 ]
Krb5Context.wrap: data=[01 01 00 00 7a 6b 63 6c 69 40 49 42 4d 2e 43 4f 4d ]
Krb5Context.wrap: token=[60 43 06 09 2a 86 48 86 f7 12 01 02 02 02 01 00 00 ff ff ff ff b7 a7 7d 39 a5 2e a3 82 24 50 6b 54 b1 92 ce 19 7d 01 ae 93 8b a9 60 1c 01 01 00 00 7a 6b 63 6c 69 40 49 42 4d 2e 43 4f 4d 07 07 07 07 07 07 07 ]
13/08/09 15:53:21 ERROR zookeeper.ClientCnxn: Error while calling watcher
java.lang.IllegalStateException: Received event is not valid.
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:372)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:283)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)

 

Seems it a bug https://issues.apache.org/jira/browse/HBASE-8757

 

I am not sure cloudera 4.3.0 using the fixed version of Hbase 

3 REPLIES 3

Re: hbase client error when using kerberos

Rising Star

I am not sure cloudera 4.3.0 using the fixed version of Hbase 


CDH 4.3 includes Apache HBase hbase-0.94.6+96

 

"The following lists all Apache HBase Jiras included in CDH 4 that are not included in the Apache HBase base version 0.94.6."

http://archive.cloudera.com/cdh4/cdh/4/hbase-0.94.6-cdh4.3.0.releasenotes.html

 

 

 

HBASE-8757 is not listed.

 

"The hbase-0.94.6-cdh4.3.0.CHANGES.txt file lists all changes included in CDH 4. "

 

HBASE-8757 is not listed.

 

Which implies HBASE-8757 is not fixed in CDH 4.3.

Highlighted

Re: hbase client error when using kerberos

Rising Star

HBASE-8757 is fixed in HBase 0.98.00.95.2

Install HBase 0.98.0, or  0.95.2 

Re: hbase client error when using kerberos

Contributor

HBASE-8757 is a nice-to-have so it is not fixed in CDH4.3.0.  It's a warning and doesn't cause any function loss.

Don't have an account?
Coming from Hortonworks? Activate your account here