How to setup knox for Hive LLAP

nshelke — Fri, 04 Aug 2017 06:27:34 GMT

I have added below block in knox topology

<service>
<role>HIVE2</role>
<url>http://FQDN_LLAP_SERVER:10501/cliservice</url>
</service>

Also, created the directory in "$KNOX_HOME/data/services/hive2" with service.xml and rewrite.xml files.

Also enabled below properties in Hiveserver2-Interactive-site.xml file,

hive.server2.thrift.http.path=cliservice
hive.server2.transport.mode=http

service.xml

<service role="HIVE2" name="hive2" version="0.13.0">
    <routes>
        <route path="/hive2"/>
    </routes>
    <dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>
</service>

rewrite.xml

<rules>
    <rule dir="IN" name="HIVE2/hive2/inbound" pattern="*://*:*/**/hive2">
        <rewrite template="{$serviceUrl[HIVE2]}"/>
    </rule>
</rules>

Getting below error in knox while accessing this path from ODBC driver,

hadoop.gateway Failed to match path /hive2

Re: How to setup knox for Hive LLAP

ssubhas — Fri, 04 Aug 2017 10:17:47 GMT

@nshelke

1) Edit the Advanced topology of your KNOX service to add LLAP service

<service>
<role>LLAP</role>
<url>http://<LLAP server host>:<HTTP PORT NUMBER>/{{hive_http_path}}</url>
</service>

2) Go to the below location in your KNOX server machine:-
/usr/hdp/<HDP VERSION>/knox/data/services

3) Copy the hive directory present in the location and rename it as llap

4) Edit the services.xml and rewrite.xml as below:-

servcies.xml
------------
<service role="LLAP" name="llap" version="0.13.0">
<routes>
<route path="/llap"/>
</routes>
<dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>
</service>

<rules>
<rule dir="IN" name="LLAP/llap/inbound" pattern="*://*:*/**/llap">
<rewrite template="{$serviceUrl[LLAP]}"/>
</rule>
</rules>

Use the http path as 'gateway/default/llap' while connecting to LLAP via KNOX.

Re: How to setup knox for Hive LLAP

radoslaw_klewin — Fri, 04 Aug 2017 18:27:52 GMT

Hi,

We use your configuration, but don't work.

We don't use doAS (https://community.hortonworks.com/questions/107790/hive-interactive-and-hiveserver2enabledoas.html).

How would be create a connection string? We use this:

jdbc:hive2://<<host>>:8443/;ssl=true;transportMode=http;httpPath=gateway/default/llap;sslTrustStore=knox.jks;trustStorePassword=xxxxxx

Could you help us and any idea where is the problem?

Log from (/var/log/hive/hiveserver2Interactive.log): <pre>

2017-08-04T12:52:18,793 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(145)) - Could not validate cookie sent, will try to generate a new cookie

2017-08-04T12:52:18,825 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(398)) - Failed to authenticate with http/_HOST kerberos principal, trying with hive/_HOST kerberos principal

2017-08-04T12:52:18,828 ERROR [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(406)) - Failed to authenticate with hive/_HOST kerberos principal

2017-08-04T12:52:18,829 ERROR [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(209)) - Error:

org.apache.hive.service.auth.HttpAuthenticationException: java.lang.reflect.UndeclaredThrowableException

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:407) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:159) [hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) [javax.servlet-3.0.0.v201112011016.jar:?]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) [javax.servlet-3.0.0.v201112011016.jar:?]

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.Server.handle(Server.java:349) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:925) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:952) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]

Caused by: java.lang.reflect.UndeclaredThrowableException

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884) ~[hadoop-common-2.7.3.2.6.1.0-129.jar:?]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:404) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

... 23 more

Caused by: org.apache.hive.service.auth.HttpAuthenticationException: Authorization header received from the client is empty.

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.getAuthHeader(ThriftHttpServlet.java:548) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.access$100(ThriftHttpServlet.java:74) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:449) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:412) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101]

at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101]

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) ~[hadoop-common-2.7.3.2.6.1.0-129.jar:?]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:404) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

... 23 more

2017-08-04T12:52:18,907 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(145)) - Could not validate cookie sent, will try to generate a new cookie
2017-08-04T12:52:18,929 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(204)) - Cookie added for clientUserName knox
2017-08-04T12:52:18,950 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(310)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V10 </pre>

Re: How to setup knox for Hive LLAP

ssubhas — Sun, 06 Aug 2017 21:59:29 GMT

Hi @Radoslaw Klewin

Hive LLAP does not currently support hive.server2.enable.doAs=true, hence the error I suspect.

question Re: How to setup knox for Hive LLAP in Support Questions

How to setup knox for Hive LLAP

Re: How to setup knox for Hive LLAP

Re: How to setup knox for Hive LLAP

Re: How to setup knox for Hive LLAP