Knox POST request dropping Request Body!!

[#1]

Hi All,

I have a custom service configured behind Knox Gateway and when i issue POST Requests the target Service is not getting any Request Body Params.

Here is the config i have used for setting up a Custom Service behind Knox.

My Topology is :

<topology>
    <gateway>
        <provider>
            <role>webappsec</role>
            <name>WebAppSec</name>
            <enabled>true</enabled>
            <param>
                <name>cors.enabled</name>
                <value>true</value>
              </param>
              <param>
                    <name>cors.supportedMethods</name>
                    <value>GET,HEAD,POST,OPTIONS,PUT</value>
               </param>
                <param>
                        <name>xframe.options.enabled</name>
                        <value>true</value>
                </param>
                <param>
                        <name>xframe.options</name>
                        <value>SAMEORIGIN</value>
                </param>
        </provider>
       <provider>
            <role>identity-assertion</role>
            <name>Default</name>
            <enabled>true</enabled>
        </provider>

        <service>
            <role>MYSERVICE</role>
            <url>http://localhost:9090</url>
        </service>
</topology>

and the Service it self is described as

<service role="MYSERVICE" name="myservice" version="0.0.1">

        <policies>
            <policy role="webappsec"/>
            <policy role="authentication" name="Anonymous"/>
            <policy role="rewrite"/>
            <policy role="authorization"/>
        </policies>

        <routes>
            <route path="/myservice"/>
            <route path="/myservice/**"/>
        </routes>

        <dispatch classname="org.apache.hadoop.gateway.dispatch.PassAllHeadersDispatch"/>

</service>

and here are the rewrite Rules.

<rules>
        <rule dir="IN" name="MYSERVICE/ myservice/inbound" pattern="*://*:*/**/ myservice/{path=**}?{**}">
            <rewrite template="{$serviceUrl[MYSERVICE]}/{path=**}?{**}"/>
        </rule>


        <rule dir="IN" name="MYSERVICE/myservice/inbound/root" pattern="*://*:*/**/myservice/">
            <rewrite template="{$serviceUrl[MYSERVICE]}/"/>
        </rule>
        
        <rule dir="IN" name="MYSERVICE/myservice/inbound/path" pattern="*://*:*/**/myservice/{**}">
            <rewrite template="{$serviceUrl[MYSERVICE]}/{**}"/>
        </rule>

</rules>

When i tried access the Service GET Requests are working as expected. POST requests are also routed to the service how ever the POST Body is not getting sent to the target service. Not sure what's the issue is.

[#2]

I have enabled the DEBUG logging for http headers and http wire and i am seeing the contents of the payload (Request Body -> Form) in the logs.

2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "HTTP/1.1 200 OK[\r][\n]"
2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "Cache-Control: no-cache, no-store, must-revalidate, max-age=0[\r][\n]"
2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "Pragma: no-cache[\r][\n]"
2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "Expires: Thu, 01 Jan 1970 00:00:00 GMT[\r][\n]"
2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "Content-Type: text/html;charset=UTF-8[\r][\n]"
2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "Transfer-Encoding: chunked[\r][\n]"
2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "Date: Tue, 30 Apr 2019 17:20:37 GMT[\r][\n]"
2019-04-30 17:20:37,137 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "f4[\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9][0x9]<div>form is defined with 0 variable(s)</div>[\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9][0x9][\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9][0x9]<div>url is defined with 0 variable(s)</div>[\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9][0x9][\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[\r][\n]"
2019-04-30 17:20:37,138 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9]<form action="#" method="post">[\r][\n]"
2019-04-30 17:20:37,139 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9][0x9]<input type="text" value="value" name="value1" id="value1"/>[\r][\n]"
2019-04-30 17:20:37,139 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9][0x9]<button>Submit</button>[\r][\n]"
2019-04-30 17:20:37,139 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[0x9]</form>[\r][\n]"
2019-04-30 17:20:37,139 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[\r][\n]"
2019-04-30 17:20:37,139 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "0[\r][\n]"
2019-04-30 17:20:37,139 DEBUG http.wire (Wire.java:wire(73)) - http-outgoing-585 << "[\r][\n]"
2019-04-30 17:20:37,139 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onResponseReceived(122)) - http-outgoing-585 << HTTP/1.1 200 OK
2019-04-30 17:20:37,139 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onResponseReceived(125)) - http-outgoing-585 << Cache-Control: no-cache, no-store, must-revalidate, max-age=0
2019-04-30 17:20:37,140 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onResponseReceived(125)) - http-outgoing-585 << Pragma: no-cache
2019-04-30 17:20:37,140 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onResponseReceived(125)) - http-outgoing-585 << Expires: Thu, 01 Jan 1970 00:00:00 GMT
2019-04-30 17:20:37,140 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onResponseReceived(125)) - http-outgoing-585 << Content-Type: text/html;charset=UTF-8
2019-04-30 17:20:37,140 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onResponseReceived(125)) - http-outgoing-585 << Transfer-Encoding: chunked

Not sure whether i need an explicit rewrite rule to send the Request body to the target Service.

At least from this log it is clear that Knox is getting the Request Body.

I am pretty sure the Target Service is not getting the request body.

Appreciate any help.

Thanks

Sateesh

[#3]

Knox is sending post data as Transfer-Encoding: chunked. gUnicorn and most of wsgi server does not handle chunked request properly.
To fix this we can use nginx as proxy to handled chunked request and transmit regular request to gUnicorn.

Meanwhile a fix for this has been merged in gUnicorn codebase but we are waiting for official PYPI release with this fix