Created 08-10-2017 06:45 AM
Hi Team,
I have configured ranger and hive plugin using ambari. Following are my environment details,
HDP version = HDP-2.6.1.0
Hive version = 1.2.1000
Ranger version = 0.7.0
When i add a policy for hive, databases and tables not being autocompleted. I am getting following error in /var/log/ranger/admin/xa_portal.log.
From logs i found that it is due to hive jdbc version mismatching. https://issues.apache.org/jira/browse/HIVE-6050
Kindly help me to resolve this issue.
2017-08-10 06:17:31,323 [timed-executor-pool-0-SendThread(ip-172-31-14-251.us-east-2.compute.internal:2181)] WARN org.apache.zookeeper.ClientCnxn$SendThread (ClientCnxn.java:957) - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/dev/null'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-08-10 06:17:31,338 [timed-executor-pool-0-EventThread] ERROR org.apache.curator.ConnectionState (ConnectionState.java:244) - Authentication failed 2017-08-10 06:17:31,429 [timed-executor-pool-0] ERROR org.apache.hive.jdbc.HiveConnection (HiveConnection.java:588) - Error opening session org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79) at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:168) at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:155) at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:578) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:188) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.ranger.services.hive.client.HiveClient.initConnection(HiveClient.java:713) at org.apache.ranger.services.hive.client.HiveClient.access$100(HiveClient.java:55) at org.apache.ranger.services.hive.client.HiveClient$2.run(HiveClient.java:97) at org.apache.ranger.services.hive.client.HiveClient$2.run(HiveClient.java:95) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.ranger.services.hive.client.HiveClient.initHive(HiveClient.java:95) at org.apache.ranger.services.hive.client.HiveClient.<init>(HiveClient.java:76) at org.apache.ranger.services.hive.client.HiveConnectionMgr$1.call(HiveConnectionMgr.java:58) at org.apache.ranger.services.hive.client.HiveConnectionMgr$1.call(HiveConnectionMgr.java:55) at org.apache.ranger.plugin.util.TimedEventUtil.timedTask(TimedEventUtil.java:40) at org.apache.ranger.services.hive.client.HiveConnectionMgr.getHiveConnection(HiveConnectionMgr.java:62) at org.apache.ranger.services.hive.client.HiveResourceMgr.getHiveResources(HiveResourceMgr.java:109) at org.apache.ranger.services.hive.RangerServiceHive.lookupResource(RangerServiceHive.java:81) at org.apache.ranger.biz.ServiceMgr$LookupCallable.actualCall(ServiceMgr.java:542) at org.apache.ranger.biz.ServiceMgr$LookupCallable.actualCall(ServiceMgr.java:526) at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:508) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2017-08-10 06:17:31,430 [timed-executor-pool-0] WARN org.apache.hive.jdbc.HiveConnection (HiveConnection.java:192) - Failed to connect to ip-172-31-11-110.us-east-2.compute.internal:10000 2017-08-10 06:17:31,433 [timed-executor-pool-0-SendThread(ip-172-31-14-251.us-east-2.compute.internal:2181)] WARN org.apache.zookeeper.ClientCnxn$SendThread (ClientCnxn.java:957) - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/dev/null'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2017-08-10 06:17:31,433 [timed-executor-pool-0-EventThread] ERROR org.apache.curator.ConnectionState (ConnectionState.java:244) - Authentication failed 2017-08-10 06:17:31,442 [timed-executor-pool-0] WARN org.apache.hive.jdbc.HiveConnection (HiveConnection.java:213) - Could not open client transport with JDBC Uri: jdbc:hive2://ip-172-31-11-110.us-east-2.compute.internal:10000/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: Could not establish connection to jdbc:hive2://ip-172-31-11-110.us-east-2.compute.internal:10000/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) Retrying 0 of 1 2017-08-10 06:17:31,444 [timed-executor-pool-0] ERROR org.apache.hive.jdbc.HiveConnection (HiveConnection.java:588) - Error opening session org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default})
Created 09-13-2017 01:56 PM
HI Th Kr,
i did following steps,
1. did backup of this folder /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive in ranger node.
cp /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hivebkp 1. copy hive jars from another node.
2. copied hive 1.2 jars and its dependencies jars into /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive folder
scp -r /usr/hdp/2.6.1.0-129/hive/lib/* root@hostname:/usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive/ where ip-172-31-4-145.us-east-2.compute.internal is ranger node. change ownership in ranger node.
3. Copied ranger plugin into /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive/ folder
cp ../hivebkp/ranger-hive-plugin-0.7.0.2.6.1.0-129.jar /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive/
3. Changed ownership of this folder
chown -R ranger:ranger /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive
4. restarted ranger service and tested connection.
Note: this error not happened in hdp-2.6.2
Created 08-11-2017 09:04 AM
I fixed this issue by changing ranger hive plugin classpath into hive-1.2 jdbc jar and its dependencies.
Best Regards,
Mahesh Sankaran
Created 09-12-2017 03:42 PM
Hey,
could give a little more information where you exactly changed the classpath?
We're stuck at the same problem.
Thanks!
Created 09-13-2017 01:56 PM
HI Th Kr,
i did following steps,
1. did backup of this folder /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive in ranger node.
cp /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hivebkp 1. copy hive jars from another node.
2. copied hive 1.2 jars and its dependencies jars into /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive folder
scp -r /usr/hdp/2.6.1.0-129/hive/lib/* root@hostname:/usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive/ where ip-172-31-4-145.us-east-2.compute.internal is ranger node. change ownership in ranger node.
3. Copied ranger plugin into /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive/ folder
cp ../hivebkp/ranger-hive-plugin-0.7.0.2.6.1.0-129.jar /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive/
3. Changed ownership of this folder
chown -R ranger:ranger /usr/hdp/2.6.1.0-129/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive
4. restarted ranger service and tested connection.
Note: this error not happened in hdp-2.6.2
Created 09-14-2017 01:04 PM
Thanks! With HDP 2.6.2 it works fine!