Getting RegionOpeningException while trying to run a phoenix query from zeppelin, what does it mean?

kbadani — Fri, 24 Mar 2017 01:19:49 GMT

I have a secured cluster with HBase, phoenix and zeppelin installed. I have configured my jdbc(phoenix) interpreter as follows

phoenix.password

phoenix.url jdbc:phoenix:host-1,host-2,host-3:/hbase-secure

phoenix.user phoenixuser

zeppelin.jdbc.auth.type KERBEROS

zeppelin.jdbc.keytab.location /path/to/keytab/zeppelin.server.kerberos.keytab

zeppelin.jdbc.principal zeppelinxxxxxx@EXAMPLE.COM

I am trying to run a phoenix query as follows

%jdbc(phoenix)

create table if not exists PRICES ( SYMBOL varchar(10), DATE varchar(10), TIME varchar(10), OPEN varchar(10), HIGH varchar(10), LOW varchar(10), CLOSE varchar(10), VOLUME varchar(30), CONSTRAINT pk PRIMARY KEY (SYMBOL, DATE, TIME) )

I am getting following exception:

org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=1, exceptions:
Thu Mar 23 18:07:58 UTC 2017, RpcRetryingCaller{globalStartTime=1490292478525, pause=100, retries=1}, org.apache.hadoop.hbase.exceptions.RegionOpeningException: org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region SYSTEM.CATALOG,,1490292472268.e77aa4c58fd9ca2a3c7881bd081c826f. is opening on host-3,16020,1490291571015
	at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3059)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1007)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1942)
	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)


	at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:416)
	at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:564)
	at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:489)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
	at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

what does RegionOpeningException indicate?

All my region servers are alive and running, also I am able to create table using hbase shell

hbase(main):001:0> whoami

hbase@EXAMPLE.COM (auth:KERBEROS)

    groups: hadoop, hbase


hbase(main):002:0> create 't1','f1', 'f2', 'f3'

0 row(s) in 2.6210 seconds

=> Hbase::Table - t1

hbase(main):003:0>

Re: Getting RegionOpeningException while trying to run a phoenix query from zeppelin, what does it mean?

deepesh1 — Fri, 24 Mar 2017 03:31:25 GMT

Can you check if there are region-level inconsistencies? To recover try as hbase user

hbase hbck -fix

question Getting RegionOpeningException while trying to run a phoenix query from zeppelin, what does it mean? in Support Questions

Getting RegionOpeningException while trying to run a phoenix query from zeppelin, what does it mean?

Re: Getting RegionOpeningException while trying to run a phoenix query from zeppelin, what does it mean?