Member since
11-11-2016
6
Posts
1
Kudos Received
0
Solutions
11-11-2016
05:24 PM
Hey, Josh, thanks a lot for the responses -- I went through http://apache-hbase.679495.n3.nabble.com/DISCUSS-More-Shading-td4083025.html and ended up at exactly HBASE-15638 which you give. Looks it will take a while -- quoting from the first link: The HBASE-15638 (protobuf shading) approach has us reference the relocated artifact explicitly. This makes for an ugly ripple across the codebase as we declare which protobuf Message is intended; either com.google.protobuf.Message or org.apache.hadoop.hbase.shaded.com.google.protobuf.Message. It is a pain making all the changes but the intent is clear. So that means that having the gRPC and HBase stuff in the same process is a no-go for now... definitely strange. Thank you once again, I have upvoted all your answers.
... View more
11-11-2016
04:57 PM
Thanks for the response -- even before I saw your message, I happily found out myself that there is a shaded version -- but *unfortunately*, yes, it does *not* work around protobuf versions. Can I raise a HDP bug about that? I mean, this is strange to me -- I mean -- does not anyone else use HBase java client with gRPC yet? Sorry for being a bit bothersome on that, it's really strange to me, honestly, as both projects are not something new.
... View more
11-11-2016
04:11 PM
ah, so hbase-protocol actually has to do with protobuf stuff... didn't realize that. And yes, you recall correctly -- https://www.versioneye.com/java/org.apache.hbase:hbase-protocol/1.1.2 shows that protobuf-java is version 2.5 for HBase 1.1.2. What can be done about it? Given that I can control neither HDP libs, nor gRPC libs, I guess shading or a similar approach won't work, correct?
... View more
11-11-2016
03:48 PM
1 Kudo
Josh, I discovered something. Because I had the following gRPC references in the Gradle: compile 'io.grpc:grpc-netty:1.0.1'
compile 'io.grpc:grpc-protobuf:1.0.1'
compile 'io.grpc:grpc-stub:1.0.1' and now when I commented them out, the whole issue disappears (i.e. is solved). So it seems to be a clash of something, but it's strange -- at first sight, of course gRPC can't be containing references to Hadoop/Hbase classes.
... View more
11-11-2016
03:39 PM
Yes, it's there: here's the actual command executed: C:\Users\testUser\.gradle\caches\modules-2\files-2.1\org.apache.hbase\hbase-protocol\1.1.2.2.5.0.0-1245\b037b1aff481f7dfc40f918a38005a6dce72fe49>jar tf hbase-protocol-1.1.2.2.5.0.0-1245.jar| findstr ByteStringer
org/apache/hadoop/hbase/util/ByteStringer.class
I also had deleted the entire .gradle cache earlier, and everything got downloaded again, and still the same. And yes, I'm using gradle run, not some other external way which relies on a different classpath.
... View more
11-11-2016
01:58 PM
I'm getting the following error with HDP 2.5 client libraries for Hbase. I use them via Gradle, having a line like that: compile group: 'org.apache.hbase', name: 'hbase-client', version: '1.1.2.2.5.0.0-1245' Then, upon running the app, again via Gradle, I do see hbase-protocols explicitly referred in the Zookeeper client classpath, , i.e. the line I see starting with 6879 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path= contains, among other JARs, a reference to the Gradle cache, i.e. C:\Users\testUser\.gradle\caches\modules-2\files-2.1\org.apache.hbase\hbase-protocol\1.1.2.2.5.0.0-1245\b037b1aff481f7dfc40f918a38005a6dce72fe49\hbase-protocol-1.1.2.2.5.0.0-1245.jar So basically org.apache.hadoop.hbase.util.ByteStringer should be found. But it is not: 13640 [main-SendThread(hadoop-dev:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x15853747d08002b, packet:: clientPath:null serverPath:null finished:false header:: 12,4 replyHeader:: 12,1530,0 request:: '/hbase-unsecure/meta-region-server,F response:: #ffffffff0001a726567696f6e7365727665723a3136303230ffffffff7653ffffffe44971ffffffa3ffffffcf50425546a23a1773616e64626f782e686f72746f6e776f726b732e636f6d10ffffff947d18ffffffc2ffffffaeffffff93ffffff9cffffff852b100183,s{1378,1378,1478870023688,1478870023688,0,0,0,0,76,0,1378}
14143 [main] DEBUG org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation - locateRegionInMeta parentTable=hbase:meta, metaLocation=, attempt=4 of 35 failed; retrying after sleep of 1005 because: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ByteStringer
14146 [main-SendThread(hadoop-dev:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x15853747d08002b, packet:: clientPath:null serverPath:null finished:false header:: 13,8 replyHeader:: 13,1530,0 request:: '/hbase-unsecure,F response:: v{'meta-region-server,'rolllog-proc,'backup-masters,'switch,'table,'draining,'region-in-transition,'table-lock,'running,'master,'namespace,'hbaseid,'online-snapshot,'replication,'splitWAL,'recovering-regions,'rs,'flush-table-proc}
As soon as I change my Gradle hbase-client reference to: compile group: 'org.apache.hbase', name: 'hbase-client', version: '1.1.2' i.e. the library becomes the default, non-recommended HBase libs, the problem goes away. But, of course, I do not want to use the non-recommended HBase client libraries - I had another issue that actually got resolved by using the Hortonworks libs.
... View more
Labels:
- Labels:
-
Apache HBase