Created 09-19-2017 08:37 PM
We are querying HS2 using knox through beeline and also other jdbc tool and getting frequent disconnection. Below is the url for connection thorugh beeline:-
jdbc:hive2://c3master03-nn.abc.org:8445/;ssl=true?hive.server2.transport.mode=http;hive.server2.thrift.http.path=gateway/default/hive
After connection if i do not query for a minute then i got below error(Same with Squirrel JDBC client as well)
Getting log thread is interrupted, since query is done! Error: org.apache.thrift.transport.TTransportException: org.apache.http.NoHttpResponseException: c3master03-nn.abc.org:8445 failed to respond (state=08S01,code=0) java.sql.SQLException: org.apache.thrift.transport.TTransportException: org.apache.http.NoHttpResponseException: c3master03-nn.abc.org:8445 failed to respond at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:305) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:238) at org.apache.hive.beeline.Commands.execute(Commands.java:863) at org.apache.hive.beeline.Commands.sql(Commands.java:728) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:993) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:833) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:791) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:491) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:474) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: org.apache.thrift.transport.TTransportException: org.apache.http.NoHttpResponseException: c3master03-nn.abc.org:8445 failed to respond at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:297) at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) at org.apache.hive.service.cli.thrift.TCLIService$Client.send_ExecuteStatement(TCLIService.java:223) at org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:215) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.jdbc.HiveConnection$SynchronizedHandler.invoke(HiveConnection.java:1363) at com.sun.proxy.$Proxy0.ExecuteStatement(Unknown Source) at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:296) ... 14 more Caused by: org.apache.http.NoHttpResponseException: c3master03-nn.abc.org:8445 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:84) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:117) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251) ... 26 more
Even After this exception , if i rerun the query on same beeline window , it will get executed and show me the result.
And after the wait for 1 minute if i execute the same query or any other query , same exception for one time and on rerun result is there...What is this weird behavior..
Even the below properties have sufficient values.
Can some one help what is the issue or configuration changes required...
Created 09-20-2017 02:23 AM
It seems to be a knox bug https://issues.apache.org/jira/browse/KNOX-890
Workaround:-
Append "http.header.Connection=close" in the JDBC connection string.
For example, with Beeline, use the following command:
beeline -u "jdbc:hive2://sandbox.hortonworks.com:8443/ ;ssl=false;sslTrustStore=/tmp/myNewTrustStore.jks;trustStorePassword=changeit;transportMode =http;httpPath=gateway/default/hive;http.header.Connection=close" -n admin -p admin-password
Created 09-20-2017 06:58 AM
@Rajesh...
Thanks ...It is working for beeline.
Since it is a bug in Knox , can we upgrade from KNOX 0.9 to Knox 0.12 for HDP2.5.3 ? is there any document for that as i was not able to find any doc for upgrading knox ?
Created 09-21-2017 08:51 PM
@Anwaar Siddiqui Great it works for you. Please accept the answer.
You can't just upgrade one component in the stack. You have to consider moving to HDP-2.6.X latest version which has Knox 0.12. Hope this helps.