Member since
09-26-2018
5
Posts
1
Kudos Received
0
Solutions
09-26-2018
04:36 AM
I do kinit on my driver node (edge node), and it works. Do I need to make sure that my session is exist on each data node? How can I make sure that. I don't ever do this thing yet. But when I try to create hive JDBC connection, it works. I think it should be same from the kerberos side, if hive works then impala should be works also.
... View more
09-26-2018
01:54 AM
Hi In our cluster, impala service is working. Impala is accessible through HUE and impala-shell. But it got failed when I try to create JDBC connection in to it (Impala), through spark or talend. This is my connection string when I try it through spark : val con = DriverManager.getConnection ("jdbc:impala://myhost.supporting.mycompany.com:21050/default;AuthMech=1;KrbRealm=SUPPORTING.MYCOMPANY.COM;KrbHostFQDN=myhost.supporting.mycompany.com;KrbServiceName=impala;") and this is my connection string when I try to connect through tImpalaConnection (talend) jdbc:hive2://prodbdaast4dr.supporting.mycompany.com:21050/default;principal=impala/_HOST@SUPPORTING.MYCOMPANY.COM But I always got error like this in spark: java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][ImpalaJDBCDriver](500169) Unable to connect to server: [Cloudera][ImpalaJDBCDriver](500591) Kerberos Authentication failed.. at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source) at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source) at com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source) at com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source) at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown Source) at com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source) at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:270) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:26) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:31) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:33) at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:35) at $iwC$$iwC$$iwC$$iwC.<init>(<console>:37) at $iwC$$iwC$$iwC.<init>(<console>:39) at $iwC$$iwC.<init>(<console>:41) at $iwC.<init>(<console>:43) at <init>(<console>:45) at .<init>(<console>:49) at .<clinit>(<console>) at .<init>(<console>:7) at .<clinit>(<console>) at $print(<console>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1045) at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1326) at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:821) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:852) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:800) at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857) at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902) at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814) at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657) at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665) at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670) at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997) at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1064) at org.apache.spark.repl.Main$.main(Main.scala:31) at org.apache.spark.repl.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) Caused by: com.cloudera.impala.support.exceptions.GeneralException: [Cloudera][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][ImpalaJDBCDriver](500169) Unable to connect to server: [Cloudera][ImpalaJDBCDriver](500591) Kerberos Authentication failed.. ... 54 more Caused by: java.lang.RuntimeException: [Cloudera][ImpalaJDBCDriver](500169) Unable to connect to server: [Cloudera][ImpalaJDBCDriver](500591) Kerberos Authentication failed. at com.cloudera.impala.hivecommon.api.HiveServerPrivilegedAction.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) .............................. Caused by: com.cloudera.impala.jdbc4.internal.apache.thrift.transport.TTransportException at com.cloudera.impala.jdbc4.internal.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ------------------------------------------ And this is the error in talend: jdbc:hive2://prodbdaast4dr.supporting.mycompany.com:21050/default;principal=impala/_HOST@SUPPORTING.MYCOMPANY.COM: null at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:219) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:167) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at bda_prod.sqoopexample_0_1.sqoopExample.tImpalaConnection_1Process(sqoopExample.java:578) at bda_prod.sqoopexample_0_1.sqoopExample.tHiveConnection_1Process(sqoopExample.java:963) at bda_prod.sqoopexample_0_1.sqoopExample.runJobInTOS(sqoopExample.java:1333) at bda_prod.sqoopexample_0_1.sqoopExample.main(sqoopExample.java:1070) Caused by: org.apache.thrift.transport.TTransportException at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:277) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:195)
... View more
Labels: