How to use Knox to securely access HBase through ODBC then Phoenix Query Server?

[#1]

We have HDP 2.5.3 deployed on RedHat Linux with Phoenix Query Server (PQS) in front of Phoenix+HBase. Our plan is to use ODBC on Windows Server through Phoenix Query Server to access HBase.

Since we do NOT plan to use Kerberos yet, we plan to use Knox as the secure server to proxy the data flow, as the diagram below:

We set up the Knox for PQS as described here, http://knox.apache.org/books/knox-0-11-0/user-guide.html#Avatica. Even though this user guide is for Knox 0.11 and HDP 2.5.3 uses Knox 0.9, according to @Josh Elser fix for https://issues.apache.org/jira/browse/KNOX-844, it should work for earlier Knox.

The setup does work in our system. The traffic from ODBC hit PQS successfully. However, there is a problem, the PQS failed to parse the request because the http request coming out of ODBC is not urlencoded, but the content is urlencoded after Knox.

After tracing all the calls, it seems that it is caused by ODBC driver sending out Avatica Http request with content-type set to "application/x-www-form-urlencoded”, even though the content is NOT urlencoded at all. But, when the request passes through Knox, it comes out as urlencoded. The PQS seems not able to parse urlencoded content. It seems to take the input as it is.

In comparison, the jdbc thin driver sends out http request with content-type “application/octet-stream”, which seems to be more appropriate.

Question:

Is there any way to change the content type of ODBC driver to use content type “application/octet-stream” instead? Or is there anyway to tell Knox server not to urlencode the content even though the content type is "application/x-www-form-urlencoded”? Do prefer the first approach.

We also need help on the following questions to use Ranger control:

• How can Knox pass the authenticated user to PQS along with the proxy-ed HTTP request?

• Then How can PQS impersonate as the authenticated user to call HBase so that Ranger can control the access by that user? Since PQS runs as "hbase" and it calls PQS as "hbase", ranger control cannot be used.

Thanks for any help!

[#2 · Accepted solution]

"After tracing all the calls, it seems that it is caused by ODBC driver sending out Avatica Http request with content-type set to "application/x-www-form-urlencoded”, even though the content is NOT urlencoded at all. But, when the request passes through Knox, it comes out as urlencoded. The PQS seems not able to parse urlencoded content. It seems to take the input as it is."

This sounds like a bug in the ODBC driver. I'll have to see what I can find...

"Then How can PQS impersonate as the authenticated user to call HBase so that Ranger can control the access by that user? Since PQS runs as "hbase" and it calls PQS as "hbase", ranger control cannot be used."

This is a factor of impersonation from PQS to HBase (really, Phoenix) only being configured when Kerberos authentication is enabled. My answer to your other question on https://community.hortonworks.com/questions/83203/how-to-access-hbase-through-odbc-then-phoenix-quer... already covered this.

"How can Knox pass the authenticated user to PQS along with the proxy-ed HTTP request?"

There is some on-going work which would help enable this in Avatica https://issues.apache.org/jira/browse/CALCITE-1539

[#3]

Thank you Josh. Please let us know whether the ODBC problem can be solved. We can live with the rest if the ODBC to Knox to PQS is working. Thanks!

[#4]

Hi Josh, is there any way to setup Knox server to not urlencode (or change) the http body regardless of the content-type when http requests pass through it?

Thanks.

[#5]

Not sure, Hong, but I'm also a Knox newbie. Let me see if I can find someone with more expertise 🙂

[#6]

It sounds like this is presently not possible to disable this re-encoding. The thought is that other services actually require this to be done. If you'd like, please feel free to create a JIRA issue in the KNOX project at Apache (or I can do that). We can try to come up with an improvement for Knox in the future to prevent this problem for other services.

I'm also in talks with the ODBC devs who said that they can likely fix the driver to address this issue at the source as well.

[#7]

Thank you Josh for looking into this. It would be convenient if Knox can have a configuration switch that would preserve the http body regardless of the content-type of incoming requests, as a proxy server essentially. Please open a JIRA issue if you think it is a good feature.

Thank you very much for your help of looking into the ODBC driver issue. Hopefully the fix can come out soon.

[#8]

@Josh Elser , We seem to hit the same issue, is this resolved ?