Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Accessing Hive JDBC/webHDFS through Knox in secured cluster (kerberos)

Labels:
avatar
author-rank mliem
Expert Contributor

Created ‎10-04-2016 10:47 PM

Hello,

I am trying to access hive JDBC through Knox in a secured cluster (kerberos). When accessing them directly, it works fine.

I am able to connect with hiveserver directly in HTTP mode passing in kerbeos principal and creating kerberos ticket

beeline -u 'jdbc:hive2:/<hive_server>:10001/;transportMode=http;httpPath=cliservice;principal=hive/_HOST@DEV.COM'

and access WEBHDFS fine directly connecting to namenode:

curl -i --negotiate -u : 'http://<namenode>:50070/webhdfs/v1/?op=LISTSTATUS'

Going through Knox gateway (using sample LDAP for simplicity), I get:

curl -iku guest:guest-password -X GET 'https://<knox_gateway>:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS'

<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 401 Authentication required</title>
</head><body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /webhdfs/v1/. 

Reason:<pre>    Authentication required</pre></p><hr/><i><small>Powered by Jetty://</small></i><br/>

In gateway-audit i do see the request getting translated to the actual internal request but its returning 401.

audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable|Request method: GET

audit|WEBHDFS|guest|||authentication|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|

audit|WEBHDFS|guest|||authentication|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Groups: []

audit|WEBHDFS|guest|||authorization|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|

audit|WEBHDFS|guest|||dispatch|uri|http://<name_node>:50070/webhdfs/v1/?op=LISTSTATUS&doAs=guest|unavailable|Request method: GET

audit|WEBHDFS|guest|||dispatch|uri|http://<name_node>:50070/webhdfs/v1/?op=LISTSTATUS&doAs=guest|success|Response status: 401

audit|WEBHDFS|guest|||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response status: 401

Similarly in Hive, I can connect to hiveserver directly but when I attempt through knox I get:

16/10/04 22:31:34 [main]: ERROR jdbc.HiveConnection: Error opening sessionorg.apache.thrift.transport.TTransportException: HTTP Response code: 401

In Hive server logs:

2016-10-04 22:31:34,063 INFO [HiveServer2-HttpHandler-Pool: Thread-299]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(398)) - Failed to authenticate with http/_HOST kerberos principal, trying with hive/_HOST kerberos principal

2016-10-04 22:31:34,063 ERROR [HiveServer2-HttpHandler-Pool: Thread-299]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(406)) - Failed to authenticate with hive/_HOST kerberos principal

2016-10-04 22:31:34,064 ERROR [HiveServer2-HttpHandler-Pool: Thread-299]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(209)) - Error:

org.apache.hive.service.auth.HttpAuthenticationException: java.lang.reflect.UndeclaredThrowableException

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:407)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:159)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)

at org.eclipse.jetty.server.Server.handle(Server.java:349)

at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449)

at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:925)

at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:952)

at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)

at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)

at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)

at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.reflect.UndeclaredThrowableException

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.java:404)

... 23 more

Caused by: org.apache.hive.service.auth.HttpAuthenticationException: Authorization header received from the client is empty.

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.getAuthHeader(ThriftHttpServlet.java:548)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet.access$100(ThriftHttpServlet.java:74)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:449)

at org.apache.hive.service.cli.thrift.ThriftHttpServlet$HttpKerberosServerAction.run(ThriftHttpServlet.java:412)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)

... 24 more

FYI I have:

hadoop.proxyuser.knox.hosts=<knox_gateway>hadoop.proxyuser.knox.groups=*

Thanks for any help!

7,519 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank george_gittu
Explorer

Created ‎10-31-2016 04:33 PM

@nshetty We are also fasing the same issue that mliem is facing and doing a kinit dint help

@mliem Are you able to get pass of this error ?

View solution in original post

5,730 Views
0 Kudos
9 REPLIES 9

avatar
author-rank nshetty author-rank
Rising Star

Created ‎10-13-2016 05:53 AM

@mliem Can you try doing kinit before running curl command?

5,730 Views
0 Kudos

avatar
author-rank george_gittu
Explorer

Created ‎10-31-2016 04:33 PM

@nshetty We are also fasing the same issue that mliem is facing and doing a kinit dint help

@mliem Are you able to get pass of this error ?

5,731 Views
0 Kudos

avatar
author-rank mliem
Expert Contributor

Created ‎10-31-2016 04:40 PM

@Gerg Git

No I did not, I ended up using a different LDAP server freeipa which has been proven to integrate with kerberos and knox nicely.

I was using openldap, cloudbreak and amazon linux servers on HDP 2.5. I suspect its something related to that or the way I had installed kerberos.

What are you using?

5,730 Views
0 Kudos

avatar
author-rank george_gittu
Explorer

Created ‎10-31-2016 04:56 PM

@mliem: We are using AD. This issue we are facing in our new HDP 2.5 build. We have this one working with no issues in our previous cluster which is HDP2.3.2 version(using same AD server). I suspect its something to do with the configs. I could find the same error reported in hbase (http://gethue.com/hbase-browsing-with-doas-impersonation-and-kerberos/) search for

Caused by: org.apache.hadoop.hbase.thrift.HttpAuthenticationException: Authorization header received from the client is empty.

5,730 Views
0 Kudos

avatar
author-rank sjogle-c
New Contributor

Created ‎10-02-2020 04:05 PM

I am not using Kerberos. I am using username/password as method for auth. But I am getting the same exact error: 

 

[Cloudera][ThriftExtension] (8) Authentication/authorization error occurred. Error details: Bad status with no error message: Unauthorized/Forbidden: Status code : 401

 

On the gateway logs, it shows:

20/10/02 18:04:34 ||2bf99023-8397-4c4c-86b1-43f5a0ab5a39|audit|HIVE||||access|uri|/gateway/default/hive|unavailable|Request method: POST
20/10/02 18:04:35 ||2bf99023-8397-4c4c-86b1-43f5a0ab5a39|audit|HIVE||||access|uri|/gateway/default/hive|success|Response status: 401

 

Any ideas?

 

 

4,757 Views
0 Kudos

avatar
author-rank 373789278
New Contributor

Created ‎11-22-2016 10:58 AM

@mliem: how did you integrate kerberos and openldap? Could you give me some advice ?

5,730 Views
0 Kudos

avatar
author-rank mister_letaz
Explorer

Created ‎05-23-2018 12:45 PM

Hi everyone, I am facing same issue connecting to Hive HTTP through Knox. On Knox gateway logs, I have:

access|uri|/gateway/default/hive/?op=LISTSTATUS|success|Response status: 401

@mliem: have you fixed your problem and if so, can you please tell me how ?

FYI, connecting directly to Hive HTTP with beeline and same crendentials works fine.

Regards.

5,730 Views
0 Kudos

avatar
author-rank Hadoop16
Contributor

Created ‎03-15-2024 09:39 AM

@mliem : Did you resolve this issue? if so can you please share what was done? I'm facing the same error when connecting to Hive via Knox.

1,424 Views
0 Kudos

avatar
author-rank DianaTorres author-rank
Community Manager

Created ‎03-15-2024 10:31 AM

@Hadoop16 Welcome to the Cloudera Community! As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks.

Regards,

Diana Torres,
Community Moderator

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
1,416 Views
0 Kudos
Announcements
What's New @ Cloudera
[RELEASED] Cloudera Streaming Analytics 1.14 for Cloudera Pu...
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
View More Announcements