Impala JDBC - Thrift Transport Exception

[#1]

Hi,

When we query Impala using JDBC connection from R or dashboard , it is throwing this error quite often.

Error message: 

org.apache.thrift.transport.TTransportException	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)	at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_FetchResults(TCLIService.java:489)	at org.apache.hive.service.cli.thrift.TCLIService$Client.FetchResults(TCLIService.java:476)	at org.apache.hive.jdbc.HiveQueryResultSet.next(HiveQueryResultSet.java:225)	at info.urbanek.Rpackage.RJDBC.JDBCResultPull.fetch(JDBCResultPull.java:77)Error in .jcall(rp, "I", "fetch", stride, block) : 
  java.sql.SQLException: Error retrieving next row

Sometimes it works fine and other times it is throwing the ThriftTransport exception. Did anyone face this issue , could anyone help me with the root cause of it and what needs to be done in this case?

Thanks,

Renuka

[#2]

Hi @renuka20,

You need to increase the memory. I mean set options for the java parameters.

For example: 

  options(java.parameters = "-Xmx8048m")

  library("RJDBC")

Regards, 

Manu.

[#3]

Hi @manuroman

These are the settings used in R for connecting to Impala , but still I am getting the same error.

  library("rJava")
  .jinit(parameters = c("-Xms6g", "-Xmx20g"))
  library("RJDBC")

Do I need to change the settings somewhere else or within R?

Thanks,

Renuka K

[#4]

Hi @renuka20

Have you 20g available? Try to restart your Rstudio.

Regards,

Manu.

[#5]

Hi @manuroman,

We are getting this error sometimes and sometimes this doesnt show up for the same query.What might be the issue , any idea?

Thanks,

Renuka