Support Questions

Find answers, ask questions, and share your expertise

Impala - SocketTimeoutException: Read timed out

avatar
New Contributor

Hello.

 

I am trying to set up a single-node cluster running Impala on Kudu with the HiveMetaStore being managed by PostgreSQL. Connecting to the database fails with a "java.net.SocketTimeoutException: Read timed out" when attempting to read from the Hive metastore through the com.cloudera.impala.jdbc41.Driver.

 

I have attempted to troubleshoot the installation using only the PostgreSQL driver, which connects correctly to the database and runs the intended workload without issues. Similarly, the tables residing on Kudu can be queried without issues from the impala-shell.

 

The installation is based on the CDH6.x-snapshot with hive-2.1.1 and the full stacktrace can be found below.

Let me know if there are other configuration options that may be of interest.

 

java.lang.RuntimeException: Failed to connect to database
        at pt.haslab.htapbench.api.Worker.<init>(Worker.java:89)
        at pt.haslab.htapbench.benchmark.TPCCWorker.<init>(TPCCWorker.java:72)
        at pt.haslab.htapbench.benchmark.HTAPBenchmark.createTerminals(HTAPBenchmark.java:189)
        at pt.haslab.htapbench.benchmark.HTAPBenchmark.makeWorkersImpl(HTAPBenchmark.java:88)
        at pt.haslab.htapbench.api.BenchmarkModule.makeWorkers(BenchmarkModule.java:209)
        at pt.haslab.htapbench.core.HTAPBench.runHybridWorkload(HTAPBench.java:846)
        at pt.haslab.htapbench.core.HTAPBench.main(HTAPBench.java:666)
Caused by: java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500151) Error setting/closing session: java.net.SocketTimeoutException: Read timed out.
        at com.cloudera.impala.hivecommon.api.HS2Client.openSession(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HS2Client.<init>(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
        at com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
        at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at pt.haslab.htapbench.api.BenchmarkModule.makeConnection(BenchmarkModule.java:106)
        at pt.haslab.htapbench.api.Worker.<init>(Worker.java:85)
        at pt.haslab.htapbench.benchmark.TPCCWorker.<init>(TPCCWorker.java:72)
        at pt.haslab.htapbench.benchmark.HTAPBenchmark.createTerminals(HTAPBenchmark.java:189)
        at pt.haslab.htapbench.benchmark.HTAPBenchmark.makeWorkersImpl(HTAPBenchmark.java:88)
        at pt.haslab.htapbench.api.BenchmarkModule.makeWorkers(BenchmarkModule.java:209)
        at pt.haslab.htapbench.core.HTAPBench.runHybridWorkload(HTAPBench.java:846)
Caused by: com.cloudera.impala.support.exceptions.GeneralException: [Cloudera][ImpalaJDBCDriver](500151) Error setting/closing session: java.net.SocketTimeoutException: Read timed out.
        ... 17 more
Caused by: com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
        at com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
        at com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at com.cloudera.impala.jdbc41.internal.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
        at com.cloudera.impala.jdbc41.internal.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
        at com.cloudera.impala.jdbc41.internal.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
        at com.cloudera.impala.jdbc41.internal.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
        at com.cloudera.impala.jdbc41.internal.apache.hive.service.cli.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:160)
        at com.cloudera.impala.hivecommon.api.HS2ClientWrapper.recv_OpenSession(Unknown Source)
        at com.cloudera.impala.jdbc41.internal.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:147)
        at com.cloudera.impala.hivecommon.api.HS2ClientWrapper.OpenSession(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HS2Client.openSession(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HS2Client.<init>(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
        at com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
        at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at pt.haslab.htapbench.api.BenchmarkModule.makeConnection(BenchmarkModule.java:106)
        at pt.haslab.htapbench.api.Worker.<init>(Worker.java:85)
        at pt.haslab.htapbench.benchmark.TPCCWorker.<init>(TPCCWorker.java:72)
        at pt.haslab.htapbench.benchmark.HTAPBenchmark.createTerminals(HTAPBenchmark.java:189)
        at pt.haslab.htapbench.benchmark.HTAPBenchmark.makeWorkersImpl(HTAPBenchmark.java:88)
        at pt.haslab.htapbench.api.BenchmarkModule.makeWorkers(BenchmarkModule.java:209)
        at pt.haslab.htapbench.core.HTAPBench.runHybridWorkload(HTAPBench.java:846)
        at pt.haslab.htapbench.core.HTAPBench.main(HTAPBench.java:666)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
        at com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
        ... 27 more
1 REPLY 1

avatar
New Contributor

The connection actually worked as intended, but I spawned more than the default number of max connections, which caused the later connections to time out. I solved the issue by starting the Impala cluster with the command: 

 

bin/start-impala-cluster.py --impalad_args="--fe_service_threads 128"