Support Questions
Find answers, ask questions, and share your expertise

authetication knox + hive + ranger

Explorer

Hi,

I'm using a full clustered installation , not the sandbox.

I have been trying to configure knox + ranger + hive but I'm stuck on authentication issues.

I have knox configured to authenticate against LDAP . It works fine for service WEBHDFS , so I believe this topology configuration is fine. I also got ranger plugin for knox configured and can manager authorization by ranger UI.

curl -iku <user>:<pwd> -X GET 'https://<knox_getway>:<port>/gateway/<topology>/webhdfs/v1/?op=LISTSTATUS'

I have hive configured to authenticate against LDAP and ranger plugin enabled. I also can access hive directly using binary or http transport mode and manager authorization by ranger UI.

ex http:

!connect jdbc:hive2://<hive_server_2>:<hive_port>/<db_name>;transportMode=http;httpPath=cliservice

Here I use cliservice as httpPath. this is teh same values defines on hive tab

But, when I try to connect to Hive through knox I get "HTTP Response code: 401" . Looking into knox log the user ai authorized . I guess I have some kind of configuration problem between knox and Hive regards the user used in this process

ex: (certificate imported to beeline truststore.)

!connect jdbc:hive2://<knox_getway>:<port>/<db_name>;transportMode=http;httpPath=gateway/<topology>/hive;ssl=true

I looked into many log files but got no clue . Is kerberos required for this scenario (access hive through knox with ranger) ?

Below there are some configurations regard this:

Knox:

gateway.hadoop.kerberos.secured = false

ranger tab:

all values regards Kerberos under "Advanced ranger-admin-site" are empty.

Tab Admin->Kerberos

Kerberos security is Diasabled

I really appreciate any orientation .

8 REPLIES 8

Re: authetication knox + hive + ranger

Explorer

this is the stack trace I get when run on beeline:

org.apache.thrift.transport.TTransportException: HTTP Response code: 401 at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:262) at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) at org.apache.hive.service.cli.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:154) at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:146) at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:552) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:170) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211) at org.apache.hive.beeline.Commands.connect(Commands.java:1190) at org.apache.hive.beeline.Commands.connect(Commands.java:1086) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:989) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:832) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:790) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:490) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:473) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Error: Could not establish connection to jdbc:hive2: .....

I'm sure the user and pwd are correct because knox access for others services and also direct Hive connection work with the same credentials.

Re: authetication knox + hive + ranger

Expert Contributor

Could you please share your setting on the ranger, including the hive plugin and knox plugin? In the meantime, please check the ranger audit to see any denied action on this connection.

Re: authetication knox + hive + ranger

Explorer

ranger policy for hive:

14719-mhl4m.png

ranger policy for knox

14718-mdbpl.png

I have some other policies , but these two above are enabled and give full access. the user I'm using is included in both .

The access to Hive and Knox works individually , so I believe that policy is OK.

I just have hive and knox plugin enabled . the others are disabled.

For Ranger settings:

Knox SSO is disabled.

ranger.plugins.knox.serviceuser = knox

ranger.plugins.hive.serviceuser = hive

Ranger Tab: User Config

14723-ir7p7.png

I have created an entry for knox user in LDAP server.

Knox topology

<gateway> <provider> <role>authentication</role> <name>ShiroProvider</name> <enabled>true</enabled> <param name="main.ldapRealm" value="org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm"/> <param name="main.ldapContextFactory" value="org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory"/> <param name="main.ldapRealm.contextFactory" value="$ldapContextFactory"/> <param name="main.ldapRealm.contextFactory.url" value="ldap://<ldapServre>:<ldapPort>"/> <param name="main.ldapRealm.contextFactory.systemUsername" value="cn=admin,dc=********"/> <param name="main.ldapRealm.contextFactory.systemPassword" value="********"/> <param name="main.ldapRealm.searchBase" value="ou=Global,dc=*******"/> <param name="main.ldapRealm.userSearchAttributeName" value="uid"/> <param name="main.ldapRealm.userObjectClass" value="inetOrgPerson"/> <param name="main.ldapRealm.authorizationEnabled" value="true"/> <param name="main.ldapRealm.groupSearchBase" value="ou=Global,dc=********"/> <param name="main.ldapRealm.groupObjectClass" value="groupOfNames"/> <param name="main.ldapRealm.groupIdAttribute" value="cn"/> <param name="main.ldapRealm.memberAttribute" value="member"/> <param name="urls./**" value="authcBasic"/> </provider>

<provider> <role>authorization</role> <name>XASecurePDPKnox</name> <enabled>true</enabled> </provider> <provider> <role>identity-assertion</role> <name>Default</name> <enabled>true</enabled> </provider> </gateway>

ALL services are included too.

The ranger audit target is HDFS , so I can't see the files on ranger UI. I got an entry from hdfs file from yesterday , but some testes I did this morning is not there.

{"repoType":5,"repo":"dtp_knox","reqUser":"fernando.braganca","evtTime":"2017-04-17 16:23:46.467","resource":"knox_ranger/HIVE","resType":"service","action":"allow","result":1,"policy":14,"enforcer":"ranger-acl","cliIP":"192.168.7.47","agentHost":"*************","logType":"RangerAudit","id":"5ea42184-3b52-4cb8-9061-c7915405a920-74","seq_num":149,"event_count":1,"event_dur_ms":0,"tags":[]}

by this entry seems that knox access is allowed , but there is no entry neither for hiveServer2 or hive audit files at the same time of the entry above.

I'm trying to connect hive through knox by beeline using

!connect jdbc:hive2://<knox_getway>:<port>/<db_name>;transportMode=http;httpPath=gateway/knox_ranger/hive;ssl=true

let me know if you need any other info.


c0ept.pnge8wza.png

Re: authetication knox + hive + ranger

Explorer

After change log4j level to debug I see more info into gateway.log . Seems to me that Kerberos is required or any other configuration regards JAAS

2017-04-18 16:11:28,113 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(116)) - Received request: POST /hive 2017-04-18 16:11:28,190 DEBUG hadoop.gateway (KnoxLdapRealm.java:getUserDn(673)) - Searching from ou=Global,dc=*********** where (&(objectclass=inetOrgPerson)(uid=fernando.braganca)) scope subtree 2017-04-18 16:11:28,199 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(679)) - Computed userDn: uid=fernando.braganca,ou=Global,dc=*********** using ldapSearch for principal: fernando.braganca 2017-04-18 16:11:28,258 DEBUG hadoop.gateway (KnoxLdapRealm.java:getUserDn(673)) - Searching from ou=Global,dc=*********** where (&(objectclass=inetOrgPerson)(uid=fernando.braganca)) scope subtree 2017-04-18 16:11:28,268 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(679)) - Computed userDn: uid=fernando.braganca,ou=Global,dc=*********** using ldapSearch for principal: fernando.braganca 2017-04-18 16:11:28,273 INFO hadoop.gateway (KnoxLdapRealm.java:rolesFor(277)) - Computed roles/groups: [HADOOP.ADMIN] for principal: fernando.braganca 2017-04-18 16:11:28,338 ERROR knox.RangerPDPKnoxFilter (RangerPDPKnoxFilter.java:getKnoxSubject(205)) - Failed to get Storm server login subject javax.security.auth.login.LoginException: No LoginModules configured for com.sun.security.jgss.initiate at javax.security.auth.login.LoginContext.init(LoginContext.java:264) at javax.security.auth.login.LoginContext.<init>(LoginContext.java:348) at org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.getKnoxSubject(RangerPDPKnoxFilter.java:199) at org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.init(RangerPDPKnoxFilter.java:69) at org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.init(RangerPDPKnoxFilter.java:129) at org.apache.hadoop.gateway.GatewayFilter$Holder.getInstance(GatewayFilter.java:362) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:331) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) .....

2017-04-18 16:11:29,846 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(164)) - Rewrote URL: https://<knox_gateway>:8443/gateway/knox_ranger/hive, direction: IN via implicit rule: HIVE/hive/inbound to URL: http://<hiveServer>:10001/cliservice 2017-04-18 16:11:29,872 DEBUG hadoop.gateway (DefaultDispatch.java:executeOutboundRequest(120)) - Dispatch request: POST http://<hiveServer>:10001/cliservice?user.name=fernando.braganca 2017-04-18 16:11:29,893 DEBUG protocol.RequestAddCookies (RequestAddCookies.java:process(122)) - CookieSpec selected: default 2017-04-18 16:11:29,905 DEBUG protocol.RequestAuthCache (RequestAuthCache.java:process(76)) - Auth cache not set in the context 2017-04-18 16:11:29,907 DEBUG conn.PoolingHttpClientConnectionManager (PoolingHttpClientConnectionManager.java:requestConnection(249)) - Connection request: [route: {}->http://<hiveServer>:10001][total kept alive: 0; route allocated: 0 of 32; total allocated: 0 of 32] 2017-04-18 16:11:29,923 DEBUG conn.PoolingHttpClientConnectionManager (PoolingHttpClientConnectionManager.java:leaseConnection(282)) - Connection leased: [id: 0][route: {}->http://<hiveServer>:10001][total kept alive: 0; route allocated: 1 of 32; total allocated: 1 of 32] 2017-04-18 16:11:29,925 DEBUG execchain.MainClientExec (MainClientExec.java:execute(234)) - Opening connection {}->http://<hiveServer>:10001 2017-04-18 16:11:29,928 DEBUG conn.DefaultHttpClientConnectionOperator (DefaultHttpClientConnectionOperator.java:connect(138)) - Connecting to <hiveServer>/192.168.7.137:10001 2017-04-18 16:11:29,942 DEBUG conn.DefaultHttpClientConnectionOperator (DefaultHttpClientConnectionOperator.java:connect(145)) - Connection established 192.168.7.47:10136<->192.168.7.137:10001 2017-04-18 16:11:29,943 DEBUG execchain.MainClientExec (MainClientExec.java:execute(255)) - Executing request POST /cliservice?user.name=fernando.braganca HTTP/1.1 2017-04-18 16:11:29,943 DEBUG execchain.MainClientExec (MainClientExec.java:execute(266)) - Proxy auth state: UNCHALLENGED 2017-04-18 16:11:29,945 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onRequestSubmitted(135)) - http-outgoing-0 >> POST /cliservice?user.name=fernando.braganca HTTP/1.1 ..... 2017-04-18 16:11:29,958 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 >> "Content-Type: application/x-thrift[\r][\n]" 2017-04-18 16:11:29,958 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 >> "Authorization: Basic ZmVybmFuZG8uYnJhZ2FuY2E6Kg==[\r][\n]" 2017-04-18 16:11:29,959 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 >> "Transfer-Encoding: chunked[\r][\n]" 2017-04-18 16:11:29,959 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 >> "Host: <hiveServer>:10001[\r][\n]" ... 2017-04-18 16:11:30,078 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 << "HTTP/1.1 401 Unauthorized[\r][\n]" 2017-04-18 16:11:30,078 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 << "Content-Length: 197[\r][\n]" 2017-04-18 16:11:30,079 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 << "Server: Jetty(7.6.0.v20120127)[\r][\n]" 2017-04-18 16:11:30,079 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 << "[\r][\n]" 2017-04-18 16:11:30,079 DEBUG http.wire (Wire.java:wire(72)) - http-outgoing-0 << "Authentication Error: javax.security.sasl.AuthenticationException: LDAP Authentication failed for user [Caused by javax.naming.AuthenticationException: [LDAP: error code 49 - Invalid Credentials]][\n]" 2017-04-18 16:11:30,082 DEBUG http.headers (LoggingManagedHttpClientConnection.java:onResponseReceived(124)) - http-outgoing-0 << HTTP/1.1 401 Unauthorized

....

Re: authetication knox + hive + ranger

Expert Contributor

One more thing, have you import the certificate into the keystore on the node where you beeline is running?

Re: authetication knox + hive + ranger

Explorer

yes, I have imported the gateway certificate into java truststore used by beeline.

Hive Settings :

use SSL is disabled;

based on this piece of log , seems to me that no user is passed to hive authentication process.

javax.security.sasl.AuthenticationException: LDAP Authentication failed for user [Caused by javax.naming.AuthenticationException

Re: authetication knox + hive + ranger

Explorer

The error happens at this method:

org.apache.hive.service.auth.LdapAuthenticationProviderImpl



2017-04-24 07:52:31,253 DEBUG [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: server.session (SessionHandler.java:doScope(180)) - sessionManager=org.eclipse.jetty.server.session.HashSessionManager@837307a
2017-04-24 07:52:31,253 DEBUG [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: server.session (SessionHandler.java:doScope(181)) - session=null
2017-04-24 07:52:31,253 DEBUG [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: servlet.ServletHandler (ServletHandler.java:doScope(392)) - servlet |/cliservice|null -> org.apache.hive.service.cli.thrift.ThriftHttpServlet-1060393799
2017-04-24 07:52:31,253 DEBUG [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: servlet.ServletHandler (ServletHandler.java:doHandle(454)) - chain=null
2017-04-24 07:52:31,253 DEBUG [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:validateCookie(307)) - Received cookies: 
2017-04-24 07:52:31,254 INFO  [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(145)) - Could not validate cookie sent, will try to generate a new cookie
2017-04-24 07:52:31,352 WARN  [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: conf.HiveConf (HiveConf.java:initialize(2980)) - HiveConf of name hive.llap.client.consistent.splits does not exist
2017-04-24 07:52:31,352 DEBUG [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: auth.LdapAuthenticationProviderImpl (LdapAuthenticationProviderImpl.java:Authenticate(166)) - Connecting using DN uid=fernando.braganca,ou=Global,dc=nuvemdtp at url ldap://n461d0116e00263.nuvemdtp:389
2017-04-24 07:52:31,354 DEBUG [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: auth.LdapAuthenticationProviderImpl (LdapAuthenticationProviderImpl.java:Authenticate(184)) - Could not connect to the LDAP Server:Authentication failed for fernando.braganca
2017-04-24 07:52:31,355 ERROR [HiveServer2-HttpHandler-Pool: Thread-20241 - /cliservice?user.name=fernando.braganca]: thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(209)) - Error: 
org.apache.hive.service.auth.HttpAuthenticationException: javax.security.sasl.AuthenticationException: LDAP Authentication failed for user [Caused by javax.naming.AuthenticationException: [LDAP: error code 49 - Invalid Credentials]]




Re: authetication knox + hive + ranger

Explorer

Well, just to let know who has experienced similar problem.

I figured out that the authentication problem (HTTP 401) is related with LDAP authentication on hiveserver2 when I go through KNOX.

If I try to connect hiveserver2 (http mode) it works with user X . If I try that via knox I get HTTP 401 for the same user X .

Both flows make authentication at the class LdapAuthenticationProviderImpl in the method Authenticate , but for some reason I don't know yet , when goes through KNOX this process fails.

My workaround was to disable hiveserver2 authentication (it was setup to LDAP) . since I have hive ranger plugin enabled , this one take care of hive policies and authorization works as expected.

I also have knox-ranger plugin enabled and going through there the knox policies take care of authorizations for services.

The side effect is that authentication in hiveserver2 is open , but not the authorization , so even if someone can connect it must have authorization to perform queries. Since transport mode for hive is set to HTTP , I believe that Knox works on peripheral security and must be the unique point to access hive . The URL to hiveserver2 must be blocked to direct access.

This scenarios works for me.

If someone knows why hiveserver2 authentication fails when user come from knox (/cliservice?user.name=X) , or have any clue regards this point , please let em know.