Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH5.1 Hbase0.98 OutOfOrderScannerNextException

CDH5.1 Hbase0.98 OutOfOrderScannerNextException

Explorer

I am inserting data from Hbase into a parquet table using Impala 1.4.0. This process works fine, but every now and then it throws an exception and crashes. It is very erratic. I had no issues prior to upgrading from CDH4.6 to CDH5.1. Any suggestions?

 

Thanks,

Eugene

 

INSERT INTO parquet_table SELECT * FROM hbase_table;
WARNINGS: DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
CAUSED BY: OutOfOrderScannerNextException: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 2 But the nextCallSeq got from client: 1; request=scanner_id: 97857 number_of_rows: 1024 close_scanner: false next_call_seq: 1
        at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3088)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29497)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
        at java.lang.Thread.run(Thread.java:745)

CAUSED BY: RemoteWithExtrasException: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 2 But the nextCallSeq got from client: 1; request=scanner_id: 97857 number_of_rows: 1024 close_scanner: false next_call_seq: 1
        at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3088)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29497)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
        at java.lang.Thread.run(Thread.java:745)



Backend 1:DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
CAUSED BY: OutOfOrderScannerNextException: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 2 But the nextCallSeq got from client: 1; request=scanner_id: 97857 number_of_rows: 1024 close_scanner: false next_call_seq: 1
        at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3088)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29497)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
        at java.lang.Thread.run(Thread.java:745)

CAUSED BY: RemoteWithExtrasException: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq: 2 But the nextCallSeq got from client: 1; request=scanner_id: 97857 number_of_rows: 1024 close_scanner: false next_call_seq: 1
        at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3088)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29497)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
        at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
        at java.lang.Thread.run(Thread.java:745)

Don't have an account?
Coming from Hortonworks? Activate your account here