- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
unable to start oozie workflow using Oozie webservice API via Python requests
- Labels:
-
Apache Oozie
Created on ‎09-23-2021 03:20 AM - edited ‎09-23-2021 09:20 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi All,
I am able to launch the Oozie work flow using Oozie Webservice API via CURL utility .Like about perform HTTP GET and POST requests .
Example curl command:
curl -u guestuser:guestpwd -X POST -H "Content-Type: application/xml;charset=UTF-8" -d @Job.xml "https://knowxhost:8443/gateway/cdp-proxy-api/oozie/v1/jobs?action=start"
url: https://knowxhost:8443/gateway/cdp-proxy-api/oozie/v1/job/0000009-210910123241282-oozie-oozi-W'
Also in python , I can fetch the Oozie executed workflow.
response = requests.get(url, auth=HTTPBasicAuth('guestuser','guestpwd '), verify=False).text
But unable to perform the POST request and getting 500 Error .
url = 'https://knowxhost:8443/gateway/cdp-proxy-api/oozie/v1/jobs?action=start'
response = requests.post(url, auth=HTTPBasicAuth('guestuser','guestpwd'), verify=False, data=xml, headers=headers).text
Exception:
HTTP ERROR 500 javax.servlet.ServletException: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.
URI:STATUS:MESSAGE:SERVLET:CAUSED BY:CAUSED BY:CAUSED BY:CAUSED BY:CAUSED BY:CAUSED BY:
/gateway/cdp-proxy-api/oozie/v1/jobs |
500 |
javax.servlet.ServletException: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. |
cdp-proxy-api-knox-gateway-servlet |
javax.servlet.ServletException: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. |
javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. |
org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. |
java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. |
java.io.IOException: java.io.IOException: Service connectivity error. |
java.io.IOException: Service connectivity error. |
Detailed stackTrace:
GatewayFilter.java:doFilter(169)) - Gateway processing failed: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.
javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.
at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196)
at org.apache.shiro.web.filter.authc.AuthenticatingFilter.cleanup(AuthenticatingFilter.java:155)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:450)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.knox.gateway.filter.ResponseCookieFilter.doFilter(ResponseCookieFilter.java:49)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:50)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:167)
at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:92)
at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:135)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:228)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:106)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:115)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:279)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:383)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:389)
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter.doFilter(ShiroSubjectIdentityAdapter.java:71)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
... 77 more
Caused by: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:142)
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:74)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
... 83 more
Caused by: java.io.IOException: java.io.IOException: Service connectivity error.
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:269)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:263)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:263)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:263)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequestWrapper(ConfigurableHADispatch.java:125)
at org.apache.knox.gateway.dispatch.DefaultDispatch.doPost(DefaultDispatch.java:343)
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$PostAdapter.doMethod(GatewayDispatchFilter.java:182)
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:125)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:171)
at org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:110)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:193)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.access$000(AbstractIdentityAssertionFilter.java:53)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter$1.run(AbstractIdentityAssertionFilter.java:161)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doAs(AbstractIdentityAssertionFilter.java:156)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:146)
at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:94)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263)
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:90)
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:87)
... 90 more
Caused by: java.io.IOException: Service connectivity error.
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:188)
at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:159)
... 123 more
Can someone assist for this issue to be solved .The user has access all the hadoop components via knox.
Created ‎03-17-2022 10:37 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @VR46 ,
I did solve the issue by encoding the username and password using base64 module and passed to post request . It worked
Created ‎03-10-2022 12:18 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
In order to assist with this, I'd need to check more python code which you are running. Could you please share the snippet around the POST call?
Also, I noticed the RangerPDPKnoxFilter in the stack trace. This means that your Knox topology (cdp-proxy-api) has Ranger Knox plugin enabled for authorization. Can you please disable the plugin (only for testing) and try again?
Hope this helps. Thanks.
Created ‎03-17-2022 10:37 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @VR46 ,
I did solve the issue by encoding the username and password using base64 module and passed to post request . It worked
Created ‎03-18-2022 12:08 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @VR46 Thanks for the analysis. As It did not work initially in Python requests ,then i tried to check in Java Sprintboot Rest template and it started the Oozie workflow . I found the difference that Base64 package was needed for encoding the username and password .The same I applied in Python and finally it worked .
Created on ‎03-19-2022 10:16 PM - edited ‎03-19-2022 10:17 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you @adhishankarit for sharing this with us. This will be useful for other person as well. Many reader will get benefit from this.
Created ‎03-18-2022 02:40 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
