when hbase 1.0.0 was introduced in cdh 5.4.0 i think netty was kept at version 3 for compatibility.

but with hbase 1.2.0 in cdh 5.7.0 this doesn't seem to be the case? i see netty 4

since hadoop still uses netty 3 it is not clear to me how i can run map-red jobs over hbase anymore, since depending on hadoop and hbase (both provided) now causes trouble.

our approach so far has been to compile agains apache hadoop 2.6.0 and apache hbase 1.0.0, and then run the jobs on cdh 5.4.x with some hbase jars added to the hadoop classpath. i think the list of extra jars looked something like this:

HADOOP_CLASSPATH="$HADOOP_CLASSPATH:/usr/lib/hbase/conf:/usr/lib/hbase/hbase-client.jar:/usr/lib/hbase/hbase-common.jar:/usr/lib/hbase/hbase-protocol.jar:/usr/lib/hbase/hbase-server.jar:/usr/lib/hbase/lib/zookeeper.jar:/usr/lib/hbase/lib/htrace-core.jar:/usr/lib/hbase/hbase-hadoop-compat.jar:/usr/lib/hbase/hbase-hadoop2-compat.jar"

this no longer works. the first error is now:

Exception in thread "main" java.io.IOException: java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
	...
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
	... 25 more
Caused by: java.lang.NoClassDefFoundError: io/netty/channel/EventLoopGroup
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:274)
	at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2138)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2103)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2197)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2223)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:665)
	... 30 more
Caused by: java.lang.ClassNotFoundException: io.netty.channel.EventLoopGroup
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	... 37 more

 the missing netty jar is from netty 4 it seems. modifying the HADOOP_CLASSPATH to include /usr/lib/hbase/lib/netty-all-4.0.23.Final.jar makes this go away, but causes the job to hang with:

3 13:04:14 INFO client.RpcRetryingCaller: Call exception, tries=10, retries=35, started=68250 ms ago, cancelled=false, msg=row '' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=node04,60020,1471036308906, seqNum=0
16/08/13 13:04:34 INFO client.RpcRetryingCaller: Call exception, tries=11, retries=35, started=88434 ms ago, cancelled=false, msg=row '' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=node04,60020,1471036308906, seqNum=0
16/08/13 13:04:54 INFO client.RpcRetryingCaller: Call exception, tries=12, retries=35, started=108630 ms ago, cancelled=false, msg=row '' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=node04,60020,1471036308906, seqNum=0

i tried instead putting hbase client shaded jar on classpath at runtime, but this gives:

Exception in thread "main" java.lang.ClassCastException: org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto cannot be cast to org.apache.hadoop.hbase.shaded.com.google.protobuf.Message
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
    at com.sun.proxy.$Proxy12.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:757)

any ideas how to proceed?