Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Phoenix connection failure

New Contributor

HDP Version: 3.0.0

env: Phoenix+Kerberos

classOf[org.apache.phoenix.jdbc.PhoenixDriver]
import java.sql.DriverManager
val conn = DriverManager.getConnection("jdbc:phoenix:dev-dmp3.fengdai.org,dev-dmp4.fengdai.org,dev-dmp5.fengdai.org:2181:/hbase-secure:" + classPath + "hbase.headless.keytab:hbase-dev_dmp@DEVDIP.ORG")

produce error

Exception in thread "main" org.apache.phoenix.exception.PhoenixIOException: callTimeout=1200000, callDuration=1329436: Failed after attempts=16, exceptions:
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:47 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:49 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:03 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:23 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:33 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:34 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:54 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
 
	at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:138)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1204)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1501)
	at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2721)
	at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:1114)
	at org.apache.phoenix.compile.CreateTableCompiler$1.execute(CreateTableCompiler.java:192)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
	at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1806)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2569)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2532)
	at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2532)
	at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
	at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
	at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:270)
	at com.tairanchina.csp.dmp.examples.kerberos.phoenix.PhoenixZKMain$.main(PhoenixZKMain.scala:39)
	at com.tairanchina.csp.dmp.examples.kerberos.phoenix.PhoenixZKMain.main(PhoenixZKMain.scala)
Caused by: java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1329436: Failed after attempts=16, exceptions:
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:47 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:49 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:03 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:23 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:33 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:34 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:54 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
 
	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:159)
	at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3055)
	at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3047)
	at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:434)
	at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1105)
	... 21 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=16, exceptions:
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:45 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:46 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:47 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:49 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:12:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:03 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:23 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:33 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:13:53 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:13 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:34 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
Sun Dec 16 16:14:54 CST 2018, RpcRetryingCaller{globalStartTime=1544947965386, pause=100, maxAttempts=16}, java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed


	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145)
	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
	at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1076)
	at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:399)
	at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:437)
	at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:434)
	at org.apache.hadoop.hbase.client.RpcRetryingCallable.call(RpcRetryingCallable.java:58)
	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
	... 25 more
Caused by: java.io.IOException: Call to dev-dmp4.fengdai.org/10.200.168.6:16020 failed on local exception: java.io.IOException: Connection closed
	at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:180)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
	at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
	at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.cleanupCalls(NettyRpcDuplexHandler.java:202)
	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelInactive(NettyRpcDuplexHandler.java:210)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
	at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
	at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection closed
	... 25 more

how to fix this problem please.

1 REPLY 1

Explorer

Hi @jason

Are you using correct jars, Please check the below for reference.

https://streever.atlassian.net/wiki/spaces/HADOOP/pages/11993090/Phoenix+JDBC+Client+Setup

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.