Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Phoenix connection failure

Phoenix connection failure

New Contributor

HDP Version: 3.0.0

env: Phoenix+Kerberos

classOf[org.apache.phoenix.jdbc.PhoenixDriver]
import java.sql.DriverManager
val conn = DriverManager.getConnection("jdbc:phoenix:dev-dmp3.fengdai.org,dev-dmp4.fengdai.org,dev-dmp5.fengdai.org:2181:/hbase-secure:" + classPath + "hbase.headless.keytab:hbase-dev_dmp@DEVDIP.ORG")

produce error

Exception in thread "main" org.apache.phoenix.exception.PhoenixIOException: callTimeout=1200000, callDuration=1329436: Failed after attempts=16, exceptions:
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:47 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:49 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:03 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:23 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:33 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:34 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:54 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
 
	at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:138)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1204)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1501)
	at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2721)
	at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:1114)
	at org.apache.phoenix.compile.CreateTableCompiler$1.execute(CreateTableCompiler.java:192)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
	at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1806)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2569)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2532)
	at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2532)
	at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
	at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
	at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:270)
	at com.tairanchina.csp.dmp.examples.kerberos.phoenix.PhoenixZKMain$.main(PhoenixZKMain.scala:39)
	at com.tairanchina.csp.dmp.examples.kerberos.phoenix.PhoenixZKMain.main(PhoenixZKMain.scala)
Caused by: java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1329436: Failed after attempts=16, exceptions:
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:47 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:49 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:03 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:23 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:33 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:34 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:54 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
 
	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:159)
	at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3055)
	at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3047)
	at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:434)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1105)
	... 21 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=16, exceptions:
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:47 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:49 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:03 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:23 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:33 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:34 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:54 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed


	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145)
	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
	at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1076)
	at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:399)
	at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:437)
	at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:434)
	at org.apache.hadoop.hbase.client.RpcRetryingCallable.call(RpcRetryingCallable.java:58)
	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
	... 25 more
Caused by: java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
	at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:180)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
	at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
	at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.cleanupCalls(NettyRpcDuplexHandler.java:202)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelInactive(NettyRpcDuplexHandler.java:210)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
	at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection closed
	... 25 more

how to fix this problem please.

1 REPLY 1

Re: Phoenix connection failure

New Contributor

Hi @jason

Are you using correct jars, Please check the below for reference.

https://streever.atlassian.net/wiki/spaces/HADOOP/pages/11993090/Phoenix+JDBC+Client+Setup

Don't have an account?
Coming from Hortonworks? Activate your account here