Skip to main content
buddelflinktier
buddelflinktier
Silver ⭐️⭐️
July 8, 2019
Solved

CopyTable HBase 1.2.0 to 2.1.0

Hello,

 

I have CDH 5.16.2 with HBase 1.2.0 and a new Cluster with CDH 6.2.0 with HBase 2.1.0. I would like to use CopyTable to copy data into the new cluster. The containers give an error:

ERROR [main] org.apache.hadoop.hbase.client.AsyncProcess: Failed to get region location
org.apache.hadoop.hbase.exceptions.UnknownProtocolException: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: Is this a pre-hbase-1.0.0 or asynchbase client? Client is invoking getClosestRowBefore removed in hbase-2.0.0 replaced by reverse Scan.
at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2452)
at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:332)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1598)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1418)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1218)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:410)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:359)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:244)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.close(BufferedMutatorImpl.java:169)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:120)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:676)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.UnknownProtocolException): org.apache.hadoop.hbase.exceptions.UnknownProtocolException: Is this a pre-hbase-1.0.0 or asynchbase client? Client is invoking getClosestRowBefore removed in hbase-2.0.0 replaced by reverse Scan.
at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2452)
at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:34070)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1594)

How can I carry the data (about 80 TB)?

 

aps
apsAnswer
Gold ⭐️
September 1, 2020

There are several options available for this.  If you plan to use CopyTable, this will not work without additional configuration.

 

Communication between an older client and newer server is not guaranteed, there is currently a workaround to allow this to work by adding the following property to your client configuration.

 

On the client being used to launch the CopyTable, you can do either:

 

Command Line:

-Dhbase.meta.replicas.use=true

 

OR

 

hbase-site.xml:

 

hbase.meta.replicas.use

true

 

 

Your other option would be to use snapshots to transfer the data.

Note- If this helps, please don't forget to click on "Accept as Solution".

View in thread

1 reply

aps
Gold ⭐️
apsAnswer
Gold ⭐️
September 1, 2020

There are several options available for this.  If you plan to use CopyTable, this will not work without additional configuration.

 

Communication between an older client and newer server is not guaranteed, there is currently a workaround to allow this to work by adding the following property to your client configuration.

 

On the client being used to launch the CopyTable, you can do either:

 

Command Line:

-Dhbase.meta.replicas.use=true

 

OR

 

hbase-site.xml:

 

hbase.meta.replicas.use

true

 

 

Your other option would be to use snapshots to transfer the data.

Note- If this helps, please don't forget to click on "Accept as Solution".

Translate
    Terms & ConditionsAccessibility statement