unable to start oozie workflow using Oozie webservice API via Python requests

adhishankarit — Thu, 23 Sep 2021 16:20:07 GMT

Hi All,

I am able to launch the Oozie work flow using Oozie Webservice API via CURL utility .Like about perform HTTP GET and POST requests .

Example curl command:

curl -u guestuser:guestpwd -X POST -H "Content-Type: application/xml;charset=UTF-8" -d @Job.xml "https://knowxhost:8443/gateway/cdp-proxy-api/oozie/v1/jobs?action=start"

url: https://knowxhost:8443/gateway/cdp-proxy-api/oozie/v1/job/0000009-210910123241282-oozie-oozi-W'

Also in python , I can fetch the Oozie executed workflow.

response = requests.get(url, auth=HTTPBasicAuth('guestuser','guestpwd '), verify=False).text

But unable to perform the POST request and getting 500 Error .

url = 'https://knowxhost:8443/gateway/cdp-proxy-api/oozie/v1/jobs?action=start'

response = requests.post(url, auth=HTTPBasicAuth('guestuser','guestpwd'), verify=False, data=xml, headers=headers).text

Exception:

HTTP ERROR 500 javax.servlet.ServletException: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.

URI:STATUS:MESSAGE:SERVLET:CAUSED BY:CAUSED BY:CAUSED BY:CAUSED BY:CAUSED BY:CAUSED BY:

/gateway/cdp-proxy-api/oozie/v1/jobs

500

javax.servlet.ServletException: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.

cdp-proxy-api-knox-gateway-servlet

javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.

org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.

java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.

java.io.IOException: java.io.IOException: Service connectivity error.

java.io.IOException: Service connectivity error.

Detailed stackTrace:

GatewayFilter.java:doFilter(169)) - Gateway processing failed: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196) at org.apache.shiro.web.filter.authc.AuthenticatingFilter.cleanup(AuthenticatingFilter.java:155) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:450) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.knox.gateway.filter.ResponseCookieFilter.doFilter(ResponseCookieFilter.java:49) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:50) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:167) at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:92) at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:135) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:228) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:106) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:115) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:279) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:383) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:389) at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter.doFilter(ShiroSubjectIdentityAdapter.java:71) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) ... 77 more Caused by: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error. at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:142) at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:74) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387) ... 83 more Caused by: java.io.IOException: java.io.IOException: Service connectivity error. at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:269) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:263) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:263) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.failoverRequest(ConfigurableHADispatch.java:263) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:163) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequestWrapper(ConfigurableHADispatch.java:125) at org.apache.knox.gateway.dispatch.DefaultDispatch.doPost(DefaultDispatch.java:343) at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$PostAdapter.doMethod(GatewayDispatchFilter.java:182) at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:125) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:171) at org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:110) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:193) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.access$000(AbstractIdentityAssertionFilter.java:53) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter$1.run(AbstractIdentityAssertionFilter.java:161) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doAs(AbstractIdentityAssertionFilter.java:156) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:146) at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:94) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:349) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:263) at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:90) at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:87) ... 90 more Caused by: java.io.IOException: Service connectivity error. at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:188) at org.apache.knox.gateway.ha.dispatch.ConfigurableHADispatch.executeRequest(ConfigurableHADispatch.java:159) ... 123 more

Can someone assist for this issue to be solved .The user has access all the hadoop components via knox.

Re: unable to start oozie workflow using Oozie webservice API via Python requests

VR46 — Thu, 10 Mar 2022 08:18:11 GMT

Hello,

In order to assist with this, I'd need to check more python code which you are running. Could you please share the snippet around the POST call?
Also, I noticed the RangerPDPKnoxFilter in the stack trace. This means that your Knox topology (cdp-proxy-api) has Ranger Knox plugin enabled for authorization. Can you please disable the plugin (only for testing) and try again?

Hope this helps. Thanks.

Re: unable to start oozie workflow using Oozie webservice API via Python requests

adhishankarit — Thu, 17 Mar 2022 17:37:37 GMT

Hi @VR46 ,

I did solve the issue by encoding the username and password using base64 module and passed to post request . It worked

Re: unable to start oozie workflow using Oozie webservice API via Python requests

adhishankarit — Fri, 18 Mar 2022 07:08:18 GMT

Hi @VR46 Thanks for the analysis. As It did not work initially in Python requests ,then i tried to check in Java Sprintboot Rest template and it started the Oozie workflow . I found the difference that Base64 package was needed for encoding the username and password .The same I applied in Python and finally it worked .

Re: unable to start oozie workflow using Oozie webservice API via Python requests

zaun — Fri, 18 Mar 2022 09:40:03 GMT

Hi @VR46,

Thank you for your help. Worked for me!

Re: unable to start oozie workflow using Oozie webservice API via Python requests

VR46 — Sun, 20 Mar 2022 05:17:05 GMT

Thank you @adhishankarit for sharing this with us. This will be useful for other person as well. Many reader will get benefit from this.

Re: unable to start oozie workflow using Oozie webservice API via Python requests

Amr5 — Mon, 29 Dec 2025 13:30:51 GMT

@adhishankarit @VR46 @zaun :

i think this is also relevant to previous issue.

After getting valid kerberos ticket for the user still facing the same , Knox SSO is enforced here.

When using:

# curl -k -u "k164prda:ep8gv=rG" https://sitlxdvdlap099.saibsit.com:8443/gateway/knoxsso/api/v1/websso
-> Gives no result

# curl -k -u k164prda https://sitlxdvdlap099.saibsit.com:8443/gateway/knoxsso/api/v1/token
-> Gives no result

# curl -k -v --negotiate -u : https://sitlxdvdlap099.saibsit.com:8443/gateway/cdp-proxy/oozie/v2/admin/status
HTTP/1.1 302 Found

1-What exact Cloudera Manager configurations are required to enable Kerberos (SPNEGO) authentication for Knox API access (non-browser) or browser , so that curl --negotiate is honored instead of redirecting to KnoxSSO?

2-What are the exact prerequisites for enabling /gateway/knoxsso/api/v1/token in CDP? not able to generate token please share specific commands if you want me to try.

For our use case triggering oozie workflow using REST API from within cluster and from Informatica DEI server , or which method is supported when Knox SSO is enabled?

question Re: unable to start oozie workflow using Oozie webservice API via Python requests in Support Questions

unable to start oozie workflow using Oozie webservice API via Python requests

HTTP ERROR 500 javax.servlet.ServletException: javax.servlet.ServletException: org.apache.shiro.subject.ExecutionException: java.security.PrivilegedActionException: java.io.IOException: java.io.IOException: Service connectivity error.

Re: unable to start oozie workflow using Oozie webservice API via Python requests

Re: unable to start oozie workflow using Oozie webservice API via Python requests

Re: unable to start oozie workflow using Oozie webservice API via Python requests

Re: unable to start oozie workflow using Oozie webservice API via Python requests

Re: unable to start oozie workflow using Oozie webservice API via Python requests

Re: unable to start oozie workflow using Oozie webservice API via Python requests