question Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe in Archives of Support Questions (Read Only)

Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

james_denman — Tue, 09 Feb 2016 03:59:42 GMT

I am able to access my Hive Databases from Hive shell. I am unable to view them in Hive View and constantly receive this error:

H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

Stack trace:


    org.apache.ambari.view.hive.client.HiveClientException: H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

org.apache.ambari.view.hive.client.HiveClientException: H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe
	at org.apache.ambari.view.hive.client.Connection$3.body(Connection.java:608)
	at org.apache.ambari.view.hive.client.Connection$3.body(Connection.java:590)
	at org.apache.ambari.view.hive.client.HiveCall.call(HiveCall.java:101)
	at org.apache.ambari.view.hive.client.Connection.execute(Connection.java:590)
	at org.apache.ambari.view.hive.client.Connection.executeSync(Connection.java:629)
	at org.apache.ambari.view.hive.client.DDLDelegator.getDBListCursor(DDLDelegator.java:76)
	at org.apache.ambari.view.hive.client.DDLDelegator.getDBList(DDLDelegator.java:65)
	at org.apache.ambari.view.hive.resources.browser.HiveBrowserService.databases(HiveBrowserService.java:88)
	at sun.reflect.GeneratedMethodAccessor336.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:715)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1496)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:196)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:216)
	at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:205)
	at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:152)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:370)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
	at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe
	at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
	at org.apache.thrift.transport.TTransport.write(TTransport.java:105)
	at org.apache.thrift.transport.TSaslTransport.writeLength(TSaslTransport.java:361)
	at org.apache.thrift.transport.TSaslTransport.flush(TSaslTransport.java:469)
	at org.apache.thrift.transport.TSaslClientTransport.flush(TSaslClientTransport.java:37)
	at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:65)
	at org.apache.hive.service.cli.thrift.TCLIService$Client.send_ExecuteStatement(TCLIService.java:219)
	at org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:211)
	at org.apache.ambari.view.hive.client.Connection$3.body(Connection.java:606)
	... 97 more
Caused by: java.net.SocketException: Broken pipe
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
	at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
	... 105 more

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

pminovic — Tue, 09 Feb 2016 09:45:55 GMT

Hi @J. David, not sure but looks like something related to your Hive transport setting. What's your hive.server2.transport.mode? All versions of Ambari Hive views support binary mode, but only Ambari-2.2 and higher support http mode.

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

james_denman — Tue, 09 Feb 2016 09:56:35 GMT

@Predrag Minovic Yes it is set to Binary. Is there any problems that would come from changing to http?

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

pminovic — Tue, 09 Feb 2016 10:18:52 GMT

If your transport is "binary" then your issue is not related to transport mode. "http" mode, required for example by Knox is not supported by Ambari versions older than 2.2.0. By the way, what's your version of Ambari? I found a related question but unresolved. However, I haven't seen this in Ambari-2.1.2 and higher. Also, can you post the screen-shot of your Ambari view setup and your Hive thrift port.

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

bandarusridhar1 — Tue, 24 May 2016 07:05:53 GMT

Below steps have fixed the issue:

1. If your cluster is Kerberosed then follow below steps:

Go to Hive Service --> Configs --> General and change below property to

hive.server2.thrift.sasl.qop = auth-conf

Restart all the affected services to make sure changed are set.

Then go to Ambari Views server, Manage Ambari --> Hive View --> Setting

Hive Authentication = auth=KERBEROS;principal=hive/_HOST@EXAMPLE.COM;hive.server2.proxy.user=${username};saslQop=auth-conf

2. If your cluster is not Kerberosed the follow below steps:

Go to Hive Service --> Configs --> General and change below property to

hive.server2.thrift.sasl.qop = auth

Restart all the affected services to make sure changed are set. Then go to Ambari Views server, Manage Ambari --> Hive View --> Setting

Hive Authentication = auth=KERBEROS;principal=hive/_HOST@EXAMPLE.COM;hive.server2.proxy.user=${username};saslQop=auth

This should fix your above error message issue.

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

shihab_pri — Tue, 07 Jun 2016 11:00:30 GMT

Restart Amabri.it worked for me

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

sshimpi — Wed, 16 Nov 2016 12:53:17 GMT

Please check if postgresql is exceeded number of pool connections. Increasing the value from 100 to 500 for pool connection resolved the issue.

mysql> SHOW VARIABLES LIKE "max_connections";

Modify pool connections -

mysql> SET GLOBAL max_connections =500;

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

ian2 — Wed, 25 Jan 2017 15:01:32 GMT

Thanks. This solved my issue.

Re: Hive View Suddenly Failing with H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

shashant_panwar — Sat, 28 Jan 2017 00:29:32 GMT

Ambari restart resolved my issue...