Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to setup knox for Hive LLAP

Solved Go to solution
Highlighted

How to setup knox for Hive LLAP

I have added below block in knox topology

<service>
<role>HIVE2</role>
<url>http://FQDN_LLAP_SERVER:10501/cliservice</url>
</service>

Also, created the directory in "$KNOX_HOME/data/services/hive2" with service.xml and rewrite.xml files.

Also enabled below properties in Hiveserver2-Interactive-site.xml file,

hive.server2.thrift.http.path=cliservice
hive.server2.transport.mode=http

service.xml

<service role="HIVE2" name="hive2" version="0.13.0">
    <routes>
        <route path="/hive2"/>
    </routes>
    <dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>
</service>

rewrite.xml

<rules>
    <rule dir="IN" name="HIVE2/hive2/inbound" pattern="*://*:*/**/hive2">
        <rewrite template="{$serviceUrl[HIVE2]}"/>
    </rule>
</rules>

Getting below error in knox while accessing this path from ODBC driver,

hadoop.gateway Failed to match path /hive2
1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to setup knox for Hive LLAP

@nshelke

1) Edit the Advanced topology of your KNOX service to add LLAP service

<service>
<role>LLAP</role>
<url>http://<LLAP server host>:<HTTP PORT NUMBER>/{{hive_http_path}}</url>
</service>

2) Go to the below location in your KNOX server machine:-
/usr/hdp/<HDP VERSION>/knox/data/services

3) Copy the hive directory present in the location and rename it as llap

4) Edit the services.xml and rewrite.xml as below:-

servcies.xml
------------
<service role="LLAP" name="llap" version="0.13.0">
<routes>
<route path="/llap"/>
</routes>
<dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>
</service>

<rules>
<rule dir="IN" name="LLAP/llap/inbound" pattern="*://*:*/**/llap">
<rewrite template="{$serviceUrl[LLAP]}"/>
</rule>
</rules>

Use the http path as 'gateway/default/llap' while connecting to LLAP via KNOX.

3 REPLIES 3

Re: How to setup knox for Hive LLAP

@nshelke

1) Edit the Advanced topology of your KNOX service to add LLAP service

<service>
<role>LLAP</role>
<url>http://<LLAP server host>:<HTTP PORT NUMBER>/{{hive_http_path}}</url>
</service>

2) Go to the below location in your KNOX server machine:-
/usr/hdp/<HDP VERSION>/knox/data/services

3) Copy the hive directory present in the location and rename it as llap

4) Edit the services.xml and rewrite.xml as below:-

servcies.xml
------------
<service role="LLAP" name="llap" version="0.13.0">
<routes>
<route path="/llap"/>
</routes>
<dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>
</service>

<rules>
<rule dir="IN" name="LLAP/llap/inbound" pattern="*://*:*/**/llap">
<rewrite template="{$serviceUrl[LLAP]}"/>
</rule>
</rules>

Use the http path as 'gateway/default/llap' while connecting to LLAP via KNOX.

Re: How to setup knox for Hive LLAP

New Contributor

Hi,

We use your configuration, but don't work.

We don't use doAS (https://community.hortonworks.com/questions/107790/hive-interactive-and-hiveserver2enabledoas.html).

How would be create a connection string? We use this:

jdbc:hive2://<<host>>:8443/;ssl=true;transportMode=http;httpPath=gateway/default/llap;sslTrustStore=knox.jks;trustStorePassword=xxxxxx

Could you help us and any idea where is the problem?

Log from (/var/log/hive/hiveserver2Interactive.log): <pre>

2017-08-04T12:52:18,793 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(145)) - Could not validate cookie sent, will try to generate a new cookie

2017-08-04T12:52:18,825 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(398)) - Failed to authenticate with http/_HOST kerberos principal, trying with hive/_HOST kerberos principal

2017-08-04T12:52:18,828 ERROR [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(406)) - Failed to authenticate with hive/_HOST kerberos principal

2017-08-04T12:52:18,829 ERROR [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(209)) - Error:

org.apache.hive.service.auth.HttpAuthenticationException: java.lang.reflect.UndeclaredThrowableException

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:407) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:159) [hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) [javax.servlet-3.0.0.v201112011016.jar:?]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) [javax.servlet-3.0.0.v201112011016.jar:?]

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.Server.handle(Server.java:349) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:925) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:952) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45) [jetty-all-7.6.0.v20120127.jar:7.6.0.v20120127]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]

Caused by: java.lang.reflect.UndeclaredThrowableException

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884) ~[hadoop-common-2.7.3.2.6.1.0-129.jar:?]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:404) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

... 23 more

Caused by: org.apache.hive.service.auth.HttpAuthenticationException: Authorization header received from the client is empty.

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.getAuthHeader(ThriftHttpServlet.java:548) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.access$100(ThriftHttpServlet.java:74) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:449) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:412) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101]

at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101]

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) ~[hadoop-common-2.7.3.2.6.1.0-129.jar:?]

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:404) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]

... 23 more

2017-08-04T12:52:18,907 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(145)) - Could not validate cookie sent, will try to generate a new cookie
2017-08-04T12:52:18,929 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(204)) - Cookie added for clientUserName knox
2017-08-04T12:52:18,950 INFO [HiveServer2-HttpHandler-Pool: Thread-68]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(310)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V10 </pre>

Re: How to setup knox for Hive LLAP

Hi @Radoslaw Klewin

Hive LLAP does not currently support hive.server2.enable.doAs=true, hence the error I suspect.

Don't have an account?
Coming from Hortonworks? Activate your account here