Member since
08-10-2018
4
Posts
0
Kudos Received
0
Solutions
09-25-2018
02:43 PM
As part of our project, we need to pull data from HIVE table. We are planning to setup the HIVE Datasource in our IBM WebSphere application server to connect to the HIVE. Can you provide the steps which I need to follow to setup that? Details: HDP 2.6.5 WAS 8.5 Regards, Saravanan
... View more
Labels:
- Labels:
-
Apache Hive
08-17-2018
11:46 AM
Thanks Sindhu. The issue was caused by missing libhadoop.so file path in the java.library.path. After adding that, I am able to avoid that exception. <br>
try{
org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
org.apache.hadoop.security.UserGroupInformation.setConfiguration(conf);
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab("hive/dspprxd@ABC.XXX.COM", "/tmp/dspprxd.keytab");
// load the driver
Class.forName("org.apache.hive.jdbc.HiveDriver");
}
catch (Exception e) {
e.printStackTrace();
}
try
{
Connection con = DriverManager.getConnection("jdbc:hive2://hpchdd2-zk-1.hpc.xxx.com:2181,hpchdd2-zk-2.hpc.xxx.com:2181,hpchdd2-zk-3.hpc.xxx.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=dspprxd@ABC.XXX.COM");
}
Currently I'm getting the following exception: I'm not sure what's causing it. java.sql.SQLException: Could not create secure connection to jdbc:hive2://hpchdd2x.hpc.xxx.com:10000/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/dspprxd@ABC.XXX.COM?transportMode=http;httpPath=cliservice: Failed to open client transport at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:529) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:242) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:206) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:675) at java.sql.DriverManager.getConnection(DriverManager.java:281) at HiveJDBCKerberosClient.main(HiveJDBCKerberosClient.java:39) Caused by: javax.security.sasl.SaslException: Failed to open client transport [Caused by java.io.IOException: Could not instantiate SASL transport] at org.apache.hive.service.auth.KerberosSaslHelper.getKerberosTransport(KerberosSaslHelper.java:60) at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:484) ... 6 more Caused by: java.io.IOException: Could not instantiate SASL transport at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Client.createClientTransport(HadoopThriftAuthBridge.java:223) at org.apache.hive.service.auth.KerberosSaslHelper.getKerberosTransport(KerberosSaslHelper.java:56) ... 7 more Caused by: javax.security.sasl.SaslException: Failure to initialize security context [Caused by org.ietf.jgss.GSSException, major code: 13, minor code: 0 major string: Invalid credentials minor string: SubjectCredFinder: no JAAS Subject] at com.ibm.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:161) at com.ibm.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:79) at javax.security.sasl.Sasl.createSaslClient(Sasl.java:400) at org.apache.hive.org.apache.thrift.transport.TSaslClientTransport.<init>(TSaslClientTransport.java:72) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Client.createClientTransport(HadoopThriftAuthBridge.java:216) ... 8 more Caused by: org.ietf.jgss.GSSException, major code: 13, minor code: 0 major string: Invalid credentials
... View more
08-13-2018
06:33 PM
As part of our project, I'm trying to connect HIVE tables via JDBC. Code Snippet: Connection con = DriverManager.getConnection("jdbc:hive2://hpchd1-zk-1.abc.xxxx.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2", "hive", ""); JARS USED:
hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar hadoop-common-2.7.3.jar When I tried to establish a connection, I'm getting the following exception. I am not sure what's causing it.Please shed some light on it. Exception: Caused by: java.lang.NoSuchFieldError: org/apache/hadoop/util/PerformanceAdvisory.LOG
at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:41) ~[hadoop-common-2.7.3.jar:?]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:88) ~[?:1.8.0]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:57) ~[?:1.8.0]
at java.lang.reflect.Constructor.newInstance(Constructor.java:437) ~[?:2.6 (10-13-2016)]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:132) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:100) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:291) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:846) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:816) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge.createClientWithConf(HadoopThriftAuthBridge.java:87) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hive.service.auth.KerberosSaslHelper.getKerberosTransport(KerberosSaslHelper.java:55) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:484) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:242) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:206) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) ~[hive-jdbc-1.2.1000.2.6.5.0-292-standalone.jar:?]
at java.sql.DriverManager.getConnection(DriverManager.java:675) ~[?:1.8.0]
at java.sql.DriverManager.getConnection(DriverManager.java:258) ~[?:1.8.0]
at com.pega.pegarules.data.internal.store.rdbms.JdbcConnectionManagerImpl.getJDBCUrlConnectionWithStatus(JdbcConnectionManagerImpl.java:2082) ~[prprivate.jar:?]
at com.pega.pegarules.data.internal.store.rdbms.JdbcConnectionManagerImpl.getJDBCUrlConnection(JdbcConnectionManagerImpl.java:2031) ~[prprivate.jar:?]
at com.pega.pegarules.data.internal.access.DatabaseImpl.testConnection(DatabaseImpl.java:3806) ~[prprivate.jar:?]
at com.pega.pegarules.data.internal.access.DatabaseImpl.testConnection(DatabaseImpl.java:3777) ~[prprivate.jar:?]
at com.pegarules.generated.activity.ra_action_testdbconnection_547349b48da96b477106433930f17c4d.step3_circum0(ra_action_testdbconnection_547349b48da96b477106433930f17c4d.java:329) ~[?:?]
at com.pegarules.generated.activity.ra_action_testdbconnection_547349b48da96b477106433930f17c4d.perform(ra_action_testdbconnection_547349b48da96b477106433930f17c4d.java:113) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3597) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:646) ~[prprivate.jar:?]
... 51 more
... View more
Labels:
- Labels:
-
Apache Hive