Cloudera Community

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.
Solved Go to solution

Issue accessing WEBHDFS using Knox authentication - ERROR 500

Labels:
avatar
author-rank sparepally1
New Member

Created ‎05-24-2016 03:49 PM

Executing below curl command

curl -iku guest:guest-password -X GET 'https://hd-poc-06.unix.gsm1900.org:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS'

Throwing this error

HTTP/1.1 500 Server Error Set-Cookie: JSESSIONID=1jlx4cirafr7e1qdcrk35ixxrz;Path=/gateway/default;Secure;HttpOnly Content-Type: text/html;charset=ISO-8859-1 Cache-Control: must-revalidate,no-cache,no-store Content-Length: 1398 Server: Jetty(8.1.14.v20131031) <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/> <title>Error 500 Server Error</title> </head> <body><h2>HTTP ERROR 500</h2> <p>Problem accessing /gateway/default/webhdfs/v1/. Reason: <pre> Server Error</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> <br/> <br/>

9,147 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank sparepally1
New Member

Created ‎05-25-2016 04:00 PM

@Alex Miller @Pranay Vyas Thanks for your support.

Issue is resolved by correcting NameNode details,

<service> <role>NAMENODE</role> <url>nameservice1</url> </service>

Earlier I mentioned url as hdfs://nameservice1 updated to nameservice1.

View solution in original post

7,291 Views
0 Kudos
6 REPLIES 6

avatar
author-rank amiller
Guru

Created ‎05-24-2016 03:52 PM

Could you provide the output of the following command while executing the curl command?

tail -f /var/log/knox/gateway*.log

Also let us know the exact HDP version you're using, and whether you are using Kerberos and/or NameNode HA.

7,291 Views
0 Kudos

avatar
author-rank sparepally1
New Member

Created ‎05-24-2016 04:31 PM

@Alex Miller I see below error messages,

Caused by: java.io.IOException: java.io.IOException: Service connectivity error. at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.failoverRequest(WebHdfsHaDispatch.java:153) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.executeRequest(WebHdfsHaDispatch.java:102) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.failoverRequest(WebHdfsHaDispatch.java:147) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.executeRequest(WebHdfsHaDispatch.java:102) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.failoverRequest(WebHdfsHaDispatch.java:147) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.executeRequest(WebHdfsHaDispatch.java:102) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.failoverRequest(WebHdfsHaDispatch.java:147) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.executeRequest(WebHdfsHaDispatch.java:102) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:294) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:145) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:117) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:315) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:215) at org.apache.hadoop.gateway.filter.AclsAuthorizationFilter.doFilter(AclsAuthorizationFilter.java:89) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:315) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:215) at org.apache.hadoop.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:195) at org.apache.hadoop.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:152) at org.apache.hadoop.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:66) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:315) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:215) at org.apache.hadoop.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:315) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:215) at org.apache.hadoop.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:93) at org.apache.hadoop.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:90) ... 59 more Caused by: java.io.IOException: Service connectivity error. at org.apache.hadoop.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:140) at org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch.executeRequest(WebHdfsHaDispatch.java:92) ... 85 more

7,291 Views
0 Kudos

avatar
author-rank sparepally1
New Member

Created ‎05-24-2016 04:32 PM

@Alex Miller Yes it is NN HA enabled cluster with Kerberos authentication.

7,291 Views
0 Kudos

avatar
author-rank sparepally1
New Member

Created ‎05-24-2016 05:46 PM

2016-05-24 09:28:45,770 ERROR hadoop.gateway (GatewayServlet.java:service(126)) - Gateway processing failed: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196) at org.apache.shiro.web.filter.authc.AuthenticatingFilter.cleanup(AuthenticatingFilter.java:155) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:315) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:215) at org.apache.hadoop.gateway.filter.ResponseCookieFilter.doFilter(ResponseCookieFilter.java:38) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:315) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:215) at org.apache.hadoop.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:315) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:215) at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:129) at org.apache.hadoop.gateway.GatewayServlet.service(GatewayServlet.java:121) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:385) at org.apache.hadoop.gateway.filter.ShiroSubjectIdentityAdapter.doFilter(ShiroSubjectIdentityAdapter.java:74)

7,291 Views
0 Kudos

avatar
author-rank PranayV author-rank
Super Collaborator

Created ‎05-24-2016 09:43 PM

is ranger enabled for Knox? If so make sure you create a ranger knox policy and provide permission for guest user.

7,291 Views
0 Kudos

avatar
author-rank sparepally1
New Member

Created ‎05-25-2016 04:00 PM

@Alex Miller @Pranay Vyas Thanks for your support.

Issue is resolved by correcting NameNode details,

<service> <role>NAMENODE</role> <url>nameservice1</url> </service>

Earlier I mentioned url as hdfs://nameservice1 updated to nameservice1.

7,292 Views
0 Kudos
Announcements
What's New @ Cloudera
Announcing Cloudera Streaming Analytics 1.17: Python UDFs, S...
Community Announcements
Content Update: Evolving Our Community Content for Better, A...
Community Announcements
Product Name Updates — Community Label Changes & Notificatio...
What's New @ Cloudera
Announcing Cloudera Data Lineage for Trino
What's New @ Cloudera
Announcing Cloudera Streaming Analytics Operator for Kuberne...
View More Announcements