Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

Solved Go to solution

Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

Expert Contributor

Hello,

I am trying to access hive JDBC through Knox in a secured cluster (kerberos). When accessing them directly, it works fine.

I am able to connect with hiveserver directly in HTTP mode passing in kerbeos principal and creating kerberos ticket

beeline -u 'jdbc:hive2:/<hive_server>:10001/;transportMode=http;httpPath=cliservice;principal=hive/_HOST@DEV.COM'

and access WEBHDFS fine directly connecting to namenode:

curl -i --negotiate -u : 'http://<namenode>:50070/webhdfs/v1/?op=LISTSTATUS'

Going through Knox gateway (using sample LDAP for simplicity), I get:

curl -iku guest:guest-password -X GET 'https://<knox_gateway>:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS'

<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 401 Authentication required</title>
</head><body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /webhdfs/v1/. 

Reason:<pre>    Authentication required</pre></p><hr/><i><small>Powered by Jetty://</small></i><br/>   

In gateway-audit i do see the request getting translated to the actual internal request but its returning 401.

audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable|Request method: GET

audit|WEBHDFS|guest|||authentication|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|

audit|WEBHDFS|guest|||authentication|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Groups: []

audit|WEBHDFS|guest|||authorization|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|

audit|WEBHDFS|guest|||dispatch|uri|http://<name_node>:50070/webhdfs/v1/?op=LISTSTATUS&doAs=guest|unavailable|Request method: GET

audit|WEBHDFS|guest|||dispatch|uri|http://<name_node>:50070/webhdfs/v1/?op=LISTSTATUS&doAs=guest|success|Response status: 401

audit|WEBHDFS|guest|||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response status: 401

Similarly in Hive, I can connect to hiveserver directly but when I attempt through knox I get:

16/10/04 22:31:34 [main]: ERROR jdbc.HiveConnection: Error opening sessionorg.apache.thrift.transport.TTransportException: HTTP Response code: 401

In Hive server logs:

2016-10-04 22:31:34,063 INFO [HiveServer2-HttpHandler-Pool: Thread-299]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(398)) - Failed to authenticate with http/_HOST kerberos principal, trying with hive/_HOST kerberos principal

2016-10-04 22:31:34,063 ERROR [HiveServer2-HttpHandler-Pool: Thread-299]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(406)) - Failed to authenticate with hive/_HOST kerberos principal

2016-10-04 22:31:34,064 ERROR [HiveServer2-HttpHandler-Pool: Thread-299]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(209)) - Error:

org.apache.hive.service.auth.HttpAuthenticationException: java.lang.reflect.UndeclaredThrowableException

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:407)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:159)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)

at org.eclipse.jetty.server.Server.handle(Server.java:349)

at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449)

at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:925)

at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:952)

at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)

at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)

at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)

at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.reflect.UndeclaredThrowableException

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:404)

... 23 more

Caused by: org.apache.hive.service.auth.HttpAuthenticationException: Authorization header received from the client is empty.

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.getAuthHeader(ThriftHttpServlet.java:548)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.access$100(ThriftHttpServlet.java:74)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:449)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:412)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)

... 24 more

FYI I have:

hadoop.proxyuser.knox.hosts=<knox_gateway>hadoop.proxyuser.knox.groups=*

Thanks for any help!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

@nshetty We are also fasing the same issue that mliem is facing and doing a kinit dint help

@mliem Are you able to get pass of this error ?

View solution in original post

6 REPLIES 6
Highlighted

Re: Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

Contributor

@mliem Can you try doing kinit before running curl command?

Highlighted

Re: Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

@nshetty We are also fasing the same issue that mliem is facing and doing a kinit dint help

@mliem Are you able to get pass of this error ?

View solution in original post

Highlighted

Re: Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

Expert Contributor
@Gerg Git

No I did not, I ended up using a different LDAP server freeipa which has been proven to integrate with kerberos and knox nicely.

I was using openldap, cloudbreak and amazon linux servers on HDP 2.5. I suspect its something related to that or the way I had installed kerberos.

What are you using?

Highlighted

Re: Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

@mliem: We are using AD. This issue we are facing in our new HDP 2.5 build. We have this one working with no issues in our previous cluster which is HDP2.3.2 version(using same AD server). I suspect its something to do with the configs. I could find the same error reported in hbase (http://gethue.com/hbase-browsing-with-doas-impersonation-and-kerberos/) search for

Caused by: org.apache.hadoop.hbase.thrift.HttpAuthenticationException: Authorization header received from the client is empty.

Highlighted

Re: Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

New Contributor

@mliem: how did you integrate kerberos and openldap? Could you give me some advice ?

Highlighted

Re: Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

Hi everyone, I am facing same issue connecting to Hive HTTP through Knox. On Knox gateway logs, I have:

access|uri|/gateway/default/hive/?op=LISTSTATUS|success|Response status: 401

@mliem: have you fixed your problem and if so, can you please tell me how ?

FYI, connecting directly to Hive HTTP with beeline and same crendentials works fine.

Regards.

Don't have an account?
Coming from Hortonworks? Activate your account here