Member since
01-10-2021
8
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 338 | 03-10-2025 09:49 PM |
03-26-2025
09:57 PM
Hi All, I have been using spark 2.2 for long time in CDSW and recently trying to work in spark 3 in CDP. One of my queries is failing in spark 3 with an error of following Py4JJavaError: An error occurred while calling o96.sql. : org.apache.spark.SparkException: Cannot broadcast the table over 512000000 rows: 1235668051 rows at org.apache.spark.sql.errors.QueryExecutionErrors$.cannotBroadcastTableOverMaxTableRowsError(QueryExecutionErrors.scala:1824) Even though this same query runs fine in spark 2.2 in CDSW. My spark session configuration is following # SET GENERAL SPARK PROPERTIES # print(" Configuring General Spark Properties") spark_session_builder = spark_session_builder.appName(name="Wrangler-Routine") spark_session_builder = spark_session_builder.master(master="yarn") spark_session_builder = spark_session_builder.enableHiveSupport() spark_session_builder = spark_session_builder.config("spark.yarn.queue", "root.project") spark_session_builder = spark_session_builder.config("spark.kryoserializer.buffer", "128m") spark_session_builder = spark_session_builder.config("spark.kryoserializer.buffer.max", "2024m") # SET SPARK DRIVER PROPERTIES # print(" Configuring Spark Driver Properties") spark_session_builder = spark_session_builder.config("spark.driver.cores", "16") spark_session_builder = spark_session_builder.config("spark.driver.memory", "64g") spark_session_builder = spark_session_builder.config("spark.driver.memoryOverhead", "8g") spark_session_builder = spark_session_builder.config("spark.driver.maxResultSize", "16g") # SET SPARK EXECUTOR PROPERTIES # print(" Configuring Spark Executor Properties") spark_session_builder = spark_session_builder.config("spark.executors.instances", "16") spark_session_builder = spark_session_builder.config("spark.executor.cores", "8") spark_session_builder = spark_session_builder.config("spark.executor.memory", "8g") spark_session_builder = spark_session_builder.config("spark.executor.memoryOverhead", "8g") # SET SPARK SQL PROPERTIES # print(" Configuring Spark SQL Properties") spark_session_builder = spark_session_builder.config("spark.sql.crossJoin.enabled", "true") spark_session_builder = spark_session_builder.config("spark.sql.autoBroadcastJoinThreshold", "-1") spark_session_builder = spark_session_builder.config("spark.sql.adaptive.autoBroadcastJoinThreshold", "-1") # INSTANTIATE SPARK SESSION # print("Instantiating Spark Session") spark_session = spark_session_builder.getOrCreate() spark_session.sql("""my sql here""") what am I missing here?!
... View more
Labels:
03-10-2025
09:49 PM
2 Kudos
Sorry for my late response. I did try with the JDK version as mentioned in the driver documentation. It didn't work. However, I am now using keytab method for connecting and I am fine with it. @asish thanks a ton for all the support.
... View more
02-16-2025
10:42 PM
@asish tried with setting up sspi_bridge.dll in dbeaver.ini as mentioned in the doc. It didn't work. Is there any changes require in URL? am using krbAuthType=2.
... View more
02-11-2025
10:23 PM
Made some progress and now I am able to access hive from Dbeaver. Following changes are made. Two env variables are set pointing krb5.ini and Kerberos ticket cache file. KrbauthType set to 2 and that made the connection successful. Now, I am trying to use domain cached login information using Windows SSPI but failing. Can anyone help me on this?
... View more
02-06-2025
02:22 AM
1. yes it is same. Infect I copied it from one of the GW node of CDP cluster. 2. yes, MIT kerberos client shows the ticket is there Microsoft Windows [Version 10.0.19045.5371] (c) Microsoft Corporation. All rights reserved. C:\Users\MAMUN.SHAHEED>e: E:\>cd E:\Software\kerberos\bin E:\Software\kerberos\bin>klist Ticket cache: API:Initial default ccache Default principal: etladmin@ROBI.COM.BD Valid starting Expires Service principal 02/06/25 15:51:42 02/06/25 17:51:42 krbtgt/ROBI.COM.BD@ROBI.COM.BD renew until 02/13/25 12:51:42 E:\Software\kerberos\bin>
... View more
02-05-2025
09:03 PM
@asishthanks for replying back. Yes, I am using latest cloudera driver i.e. ClouderaHiveJDBC-2.6.25.1033. Looking in the link in your post, I tried to connect using following URL jdbc:hive2://<host.domainname>:10001/default;SSL=1;SSLTrustStore=e:\truststore.jks;SSLTrustStorePwd=<password>;LogLevel=6;LogPath=e:\t.txt;AuthMech=1;KrbRealm=<DOMAINNAME>;KrbHostFQDN=HOSTNAME.domainname;KrbServiceName=hive I copied the trustStore file in my local drive. But I am still getting the same error. My CDP configuration is kerberos+SSL+HTTP and following works fine while initiating spark session from GW servers. --conf spark.sql.hive.hiveserver2.jdbc.url="jdbc:hive2://<host.domainname>:10001;ssl=true;sslTrustStore=/opt/cloudera/security/pki/truststore.jks;SSLTrustStorePwd=<password>" Error trace log Feb 06 10:52:35.576 TRACE 62 com.cloudera.hiveserver2.jdbc.common.CommonCoreUtils.logConnectionFunctionEntrance({AuthMech=Variant[type: TYPE_WSTRING, value: 1], ConnSchema=Variant[type: TYPE_WSTRING, value: default], DatabaseType=Variant[type: TYPE_WSTRING, value: Hive], HiveServerType=Variant[type: TYPE_WSTRING, value: 2], Host=Variant[type: TYPE_WSTRING, value: host.domain], KrbHostFQDN=Variant[type: TYPE_WSTRING, value: DC1.domain], KrbRealm=Variant[type: TYPE_WSTRING, value: DOMAINNAME], KrbServiceName=Variant[type: TYPE_WSTRING, value: hive], LogLevel=Variant[type: TYPE_WSTRING, value: 6], LogPath=Variant[type: TYPE_WSTRING, value: e:\t.txt], Port=Variant[type: TYPE_WSTRING, value: 10001], SSL=Variant[type: TYPE_WSTRING, value: 1], SSLTrustStore=Variant[type: TYPE_WSTRING, value: e:\truststore.jks], SSLTrustStorePwd=Variant[type: TYPE_WSTRING, value: xxxxx]}, "Major Version: 2", "Minor Version: 6", "Hot Fix Version: 25", "Build Number: 1033", "java.vendor:Eclipse Adoptium", "java.version:21.0.5", "os.arch:amd64", "os.name:Windows 10", "os.version:10.0", "Runtime.totalMemory:113246208", "Runtime.maxMemory:2101346304", "Runtime.avaialableProcessors:8", URLClassLoader.getURLs(): No URLClassLoader available.): +++++ enter +++++ Feb 06 10:52:35.577 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalIntegerSetting("ErrMsgAndLogQueryRedactIndex", 10): +++++ enter +++++ Feb 06 10:52:35.578 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalIntegerSetting("EmulateServerProtocolVersion", -1): +++++ enter +++++ Feb 06 10:52:35.578 TRACE 62 com.cloudera.hiveserver2.dsi.core.impl.DSIConnection.getProperty(170): +++++ enter +++++ Feb 06 10:52:35.578 TRACE 62 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.readDriverSpecificSettings(): +++++ enter +++++ Feb 06 10:52:35.578 INFO 62 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.establishConnection: Catalog is set to default: Hive Feb 06 10:52:35.579 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("ExpectedFetchSize", 10000): +++++ enter +++++ Feb 06 10:52:35.579 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("BinaryColumnLength", 32767): +++++ enter +++++ Feb 06 10:52:35.579 TRACE 62 com.cloudera.hiveserver2.jdbc.core.DSCommonConnection.readIgnoreTransactionsSettings(): +++++ enter +++++ Feb 06 10:52:35.579 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("MaxConsecutiveResultFileDownloadRetries", 10): +++++ enter +++++ Feb 06 10:52:35.579 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("MaxResultFileDownloadThreads", 80): +++++ enter +++++ Feb 06 10:52:35.579 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("ResultFileDownloadTimeout", 300): +++++ enter +++++ Feb 06 10:52:35.579 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("ResultFileDownloadRetryWaitTime", 0): +++++ enter +++++ Feb 06 10:52:35.580 TRACE 62 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("FetchHeartbeatInterval", 30000): +++++ enter +++++ Feb 06 10:52:35.580 TRACE 62 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer(): +++++ enter +++++ Feb 06 10:52:35.580 DEBUG 62 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer: socketTimeout = 0, loginTimeout = 0 Feb 06 10:52:35.580 DEBUG 62 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer: SocketTimeout is: 30 seconds for test Feb 06 10:52:35.580 TRACE 62 com.cloudera.hiveserver2.hivecommon.api.ServiceDiscoveryFactory.createClient(com.cloudera.hiveserver2.hivecommon.HiveJDBCSettings@65714eed, com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnectionLogger@1dbcd228, com.cloudera.hiveserver2.jdbc.common.SWarningListener@74eeeedf): +++++ enter +++++ Feb 06 10:52:35.580 TRACE 62 com.cloudera.hiveserver2.jdbc.utils.rpc.CustomNetworkHooks.getCustomSocketFactory(com.cloudera.hiveserver2.jdbc.common.CustomNetworkHooksSettings@30ce06b4): +++++ enter +++++ Feb 06 10:52:35.580 TRACE 62 com.cloudera.hiveserver2.jdbc.rpc.CustomDnsResolverLoader.getDnsResolver(com.cloudera.hiveserver2.jdbc.common.CustomNetworkHooksSettings@30ce06b4): +++++ enter +++++ Feb 06 10:52:35.580 TRACE 62 com.cloudera.hiveserver2.jdbc.rpc.CustomDnsResolverLoader.getResolvedHost(com.cloudera.hiveserver2.jdbc.rpc.InternalDnsResolver@33559e64, "cgp-ebi-mgm01.robi.com.bd"): +++++ enter +++++ Feb 06 10:52:35.582 TRACE 62 com.cloudera.hiveserver2.hivecommon.api.TETSSLTransportFactory.getClientSocket(com.cloudera.hiveserver2.hivecommon.HiveJDBCSettings@65714eed, com.cloudera.hiveserver2.jdbc.common.SSLTransportParameters@5c3ca8c5, null, null): +++++ enter +++++ Feb 06 10:52:35.597 TRACE 62 com.cloudera.hiveserver2.jdbc.kerberos.Kerberos.getSubjectViaAccessControlContext(): +++++ enter +++++ Feb 06 10:52:35.597 TRACE 62 com.cloudera.hiveserver2.jdbc.kerberos.Kerberos.getSubjectViaJAASConfig(null): +++++ enter +++++ Feb 06 10:52:35.598 DEBUG 62 com.cloudera.hiveserver2.jdbc.kerberos.Kerberos.getSubjectViaJAASConfig: java.security.auth.login.config is not set Feb 06 10:52:35.598 TRACE 62 com.cloudera.hiveserver2.jdbc.kerberos.Kerberos.getSubjectViaJAASConfig(): +++++ enter +++++ Feb 06 10:52:35.598 TRACE 62 com.cloudera.hiveserver2.jdbc.kerberos.Kerberos.getSubjectViaTicketCache(): +++++ enter +++++ Feb 06 10:52:35.600 DEBUG 62 com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport: Kerberos subject retrieved via ticket cache lookup Feb 06 10:52:36.060 ERROR 62 com.cloudera.hiveserver2.exceptions.ExceptionConverter.toSQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed. java.sql.SQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed. at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source) at com.cloudera.hiveserver2.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source) at com.cloudera.hiveserver2.jdbc.core.LoginTimeoutConnection.connect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.AbstractDriver.connect(Unknown Source) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:109) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:83) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:214) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:133) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.openConnection(GenericDataSource.java:160) at org.jkiss.dbeaver.model.impl.jdbc.JDBCExecutionContext.connect(JDBCExecutionContext.java:124) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.initializeMainContext(JDBCRemoteInstance.java:106) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.<init>(JDBCRemoteInstance.java:61) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.initializeRemoteInstance(JDBCDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.<init>(GenericDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.meta.GenericMetaModel.createDataSourceImpl(GenericMetaModel.java:92) at org.jkiss.dbeaver.ext.generic.GenericDataSourceProvider.openDataSource(GenericDataSourceProvider.java:57) at org.jkiss.dbeaver.registry.DataSourceDescriptor.openDataSource(DataSourceDescriptor.java:1417) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect0(DataSourceDescriptor.java:1280) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect(DataSourceDescriptor.java:1070) at org.jkiss.dbeaver.runtime.jobs.ConnectJob.run(ConnectJob.java:78) at org.jkiss.dbeaver.runtime.jobs.ConnectionTestJob.run(ConnectionTestJob.java:102) at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:119) Caused by: com.cloudera.hiveserver2.support.exceptions.GeneralException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed. ... 25 more Caused by: java.lang.RuntimeException: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed at com.cloudera.hiveserver2.hivecommon.api.HiveServerPrivilegedAction.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Unknown Source) at java.base/javax.security.auth.Subject.doAs(Unknown Source) at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source) at com.cloudera.hiveserver2.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source) at com.cloudera.hiveserver2.jdbc.core.LoginTimeoutConnection.connect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.AbstractDriver.connect(Unknown Source) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:109) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:83) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:214) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:133) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.openConnection(GenericDataSource.java:160) at org.jkiss.dbeaver.model.impl.jdbc.JDBCExecutionContext.connect(JDBCExecutionContext.java:124) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.initializeMainContext(JDBCRemoteInstance.java:106) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.<init>(JDBCRemoteInstance.java:61) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.initializeRemoteInstance(JDBCDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.<init>(GenericDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.meta.GenericMetaModel.createDataSourceImpl(GenericMetaModel.java:92) at org.jkiss.dbeaver.ext.generic.GenericDataSourceProvider.openDataSource(GenericDataSourceProvider.java:57) at org.jkiss.dbeaver.registry.DataSourceDescriptor.openDataSource(DataSourceDescriptor.java:1417) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect0(DataSourceDescriptor.java:1280) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect(DataSourceDescriptor.java:1070) at org.jkiss.dbeaver.runtime.jobs.ConnectJob.run(ConnectJob.java:78) at org.jkiss.dbeaver.runtime.jobs.ConnectionTestJob.run(ConnectionTestJob.java:102) at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:119) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63) Caused by: com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TTransportException: GSS initiate failed at com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:194) at com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:272) at com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) ... 29 more
... View more
02-05-2025
02:28 AM
Hi all, I am trying to connect DBeaver from my windows 10 client PC to my CDP Hive cluster, but failing with an error message of following Caused by: com.cloudera.hiveserver2.support.exceptions.GeneralException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed. My setup CDP Version : 7.1.9 CDP OS : RHEL 9.1 MIT Kerberos Version : 4.1 DBeaver Version : 24.3.4.202502021521 KDC : Microsoft Active Directory dbeaver.ini -startup plugins/org.jkiss.dbeaver.launcher_1.0.22.202502021521.jar --launcher.library plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.2.1100.v20240722-2106 -vmargs -XX:+IgnoreUnrecognizedVMOptions -Dosgi.requiredJavaVersion=17 -Dfile.encoding=UTF-8 --add-modules=ALL-SYSTEM --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.security.util=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.jgss=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-opens=java.desktop/java.awt=ALL-UNNAMED --add-opens=java.desktop/java.awt.peer=ALL-UNNAMED --add-opens=java.sql/java.sql=ALL-UNNAMED -Xms64m -Xmx1024m -Ddbeaver.distribution.type=zip -Djava.security.krb5.debug=true -Djava.security.krb5.conf=E:\krb5.ini -Djava.security.auth.login.config="E:\jaas.conf" krb5.ini [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = false kdc_timeout = 5000 default_realm = X.X.X [realms] X.X.X.X = { kdc = DC1.x.x.x kdc = DC2.x.x.x kdc = DC3.x.x.x admin_server = x.x.x.x default_domain = x.x.x } [domain_realm] x.x.x = X.X.X .x.x.x = X.X.X [appdefaults] pam = { debug = false ticket_lifetime = 36000 renew_lifetime = 36000 forwardable = true krb4_convert = false } jaas.conf Client { com.sun.security.auth.module.Krb5LoginModule required debug=true doNotPrompt=true useTicketCache=true renewTGT=true principal="etladmin@X.X.X" ; }; Detail error log Feb 05 16:02:41.073 TRACE 44 com.cloudera.hiveserver2.jdbc.common.CommonCoreUtils.logConnectionFunctionEntrance({AuthMech=Variant[type: TYPE_WSTRING, value: 1], ConnSchema=Variant[type: TYPE_WSTRING, value: default], DatabaseType=Variant[type: TYPE_WSTRING, value: Hive], HiveServerType=Variant[type: TYPE_WSTRING, value: 2], Host=Variant[type: TYPE_WSTRING, value: cgp-ebi-mgm01.robi.com.bd], KrbAuthType=Variant[type: TYPE_WSTRING, value: 2], KrbHostFQDN=Variant[type: TYPE_WSTRING, value: DC1.robi.com.bd], KrbRealm=Variant[type: TYPE_WSTRING, value: ROBI.COM.BD], KrbServiceName=Variant[type: TYPE_WSTRING, value: hive], LogLevel=Variant[type: TYPE_WSTRING, value: 6], LogPath=Variant[type: TYPE_WSTRING, value: e:\t.txt], Port=Variant[type: TYPE_WSTRING, value: 10000]}, "Major Version: 2", "Minor Version: 6", "Hot Fix Version: 25", "Build Number: 1033", "java.vendor:Eclipse Adoptium", "java.version:21.0.5", "os.arch:amd64", "os.name:Windows 10", "os.version:10.0", "Runtime.totalMemory:82837504", "Runtime.maxMemory:1073741824", "Runtime.avaialableProcessors:8", URLClassLoader.getURLs(): No URLClassLoader available.): +++++ enter +++++ Feb 05 16:02:41.074 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalIntegerSetting("ErrMsgAndLogQueryRedactIndex", 10): +++++ enter +++++ Feb 05 16:02:41.074 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalIntegerSetting("EmulateServerProtocolVersion", -1): +++++ enter +++++ Feb 05 16:02:41.075 TRACE 44 com.cloudera.hiveserver2.dsi.core.impl.DSIConnection.getProperty(170): +++++ enter +++++ Feb 05 16:02:41.075 TRACE 44 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.readDriverSpecificSettings(): +++++ enter +++++ Feb 05 16:02:41.075 INFO 44 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.establishConnection: Catalog is set to default: Hive Feb 05 16:02:41.076 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("ExpectedFetchSize", 10000): +++++ enter +++++ Feb 05 16:02:41.076 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("BinaryColumnLength", 32767): +++++ enter +++++ Feb 05 16:02:41.076 TRACE 44 com.cloudera.hiveserver2.jdbc.core.DSCommonConnection.readIgnoreTransactionsSettings(): +++++ enter +++++ Feb 05 16:02:41.076 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("MaxConsecutiveResultFileDownloadRetries", 10): +++++ enter +++++ Feb 05 16:02:41.077 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("MaxResultFileDownloadThreads", 80): +++++ enter +++++ Feb 05 16:02:41.077 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("ResultFileDownloadTimeout", 300): +++++ enter +++++ Feb 05 16:02:41.077 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("ResultFileDownloadRetryWaitTime", 0): +++++ enter +++++ Feb 05 16:02:41.077 TRACE 44 com.cloudera.hiveserver2.hivecommon.utils.HiveCommonConnectionUtils.readOptionalPositiveIntegerSetting("FetchHeartbeatInterval", 30000): +++++ enter +++++ Feb 05 16:02:41.077 TRACE 44 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer(): +++++ enter +++++ Feb 05 16:02:41.077 DEBUG 44 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer: socketTimeout = 0, loginTimeout = 0 Feb 05 16:02:41.077 DEBUG 44 com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer: SocketTimeout is: 30 seconds for test Feb 05 16:02:41.078 TRACE 44 com.cloudera.hiveserver2.hivecommon.api.ServiceDiscoveryFactory.createClient(com.cloudera.hiveserver2.hivecommon.HiveJDBCSettings@16eeb9ce, com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnectionLogger@5c508e1, com.cloudera.hiveserver2.jdbc.common.SWarningListener@1bb3bdb): +++++ enter +++++ Feb 05 16:02:41.078 TRACE 44 com.cloudera.hiveserver2.jdbc.utils.rpc.CustomNetworkHooks.getCustomSocketFactory(com.cloudera.hiveserver2.jdbc.common.CustomNetworkHooksSettings@4ca0a914): +++++ enter +++++ Feb 05 16:02:41.078 TRACE 44 com.cloudera.hiveserver2.jdbc.rpc.CustomDnsResolverLoader.getDnsResolver(com.cloudera.hiveserver2.jdbc.common.CustomNetworkHooksSettings@4ca0a914): +++++ enter +++++ Feb 05 16:02:41.078 TRACE 44 com.cloudera.hiveserver2.jdbc.rpc.CustomDnsResolverLoader.getResolvedHost(com.cloudera.hiveserver2.jdbc.rpc.InternalDnsResolver@141d36b8, "cgp-ebi-mgm01.robi.com.bd"): +++++ enter +++++ Feb 05 16:02:41.160 TRACE 44 com.cloudera.hiveserver2.jdbc.kerberos.Kerberos.getSubjectViaTicketCache(): +++++ enter +++++ Feb 05 16:02:41.285 DEBUG 44 com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport: Kerberos subject retrieved via ticket cache lookup Feb 05 16:02:41.328 ERROR 44 com.cloudera.hiveserver2.exceptions.ExceptionConverter.toSQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed. java.sql.SQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed. at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source) at com.cloudera.hiveserver2.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source) at com.cloudera.hiveserver2.jdbc.core.LoginTimeoutConnection.connect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.AbstractDriver.connect(Unknown Source) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:109) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:83) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:214) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:133) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.openConnection(GenericDataSource.java:160) at org.jkiss.dbeaver.model.impl.jdbc.JDBCExecutionContext.connect(JDBCExecutionContext.java:124) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.initializeMainContext(JDBCRemoteInstance.java:106) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.<init>(JDBCRemoteInstance.java:61) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.initializeRemoteInstance(JDBCDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.<init>(GenericDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.meta.GenericMetaModel.createDataSourceImpl(GenericMetaModel.java:92) at org.jkiss.dbeaver.ext.generic.GenericDataSourceProvider.openDataSource(GenericDataSourceProvider.java:57) at org.jkiss.dbeaver.registry.DataSourceDescriptor.openDataSource(DataSourceDescriptor.java:1417) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect0(DataSourceDescriptor.java:1280) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect(DataSourceDescriptor.java:1070) at org.jkiss.dbeaver.runtime.jobs.ConnectJob.run(ConnectJob.java:78) at org.jkiss.dbeaver.runtime.jobs.ConnectionTestJob.run(ConnectionTestJob.java:102) at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:119) Caused by: com.cloudera.hiveserver2.support.exceptions.GeneralException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed. ... 25 more Caused by: java.lang.RuntimeException: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed at com.cloudera.hiveserver2.hivecommon.api.HiveServerPrivilegedAction.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Unknown Source) at java.base/javax.security.auth.Subject.doAs(Unknown Source) at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source) at com.cloudera.hiveserver2.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connectToServer(Unknown Source) at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source) at com.cloudera.hiveserver2.jdbc.core.LoginTimeoutConnection.connect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source) at com.cloudera.hiveserver2.jdbc.common.AbstractDriver.connect(Unknown Source) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:109) at org.jkiss.dbeaver.model.impl.jdbc.JDBCConnectionOpener.run(JDBCConnectionOpener.java:83) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:214) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:133) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.openConnection(GenericDataSource.java:160) at org.jkiss.dbeaver.model.impl.jdbc.JDBCExecutionContext.connect(JDBCExecutionContext.java:124) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.initializeMainContext(JDBCRemoteInstance.java:106) at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.<init>(JDBCRemoteInstance.java:61) at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.initializeRemoteInstance(JDBCDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.<init>(GenericDataSource.java:125) at org.jkiss.dbeaver.ext.generic.model.meta.GenericMetaModel.createDataSourceImpl(GenericMetaModel.java:92) at org.jkiss.dbeaver.ext.generic.GenericDataSourceProvider.openDataSource(GenericDataSourceProvider.java:57) at org.jkiss.dbeaver.registry.DataSourceDescriptor.openDataSource(DataSourceDescriptor.java:1417) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect0(DataSourceDescriptor.java:1280) at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect(DataSourceDescriptor.java:1070) at org.jkiss.dbeaver.runtime.jobs.ConnectJob.run(ConnectJob.java:78) at org.jkiss.dbeaver.runtime.jobs.ConnectionTestJob.run(ConnectionTestJob.java:102) at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:119) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63) Caused by: com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TTransportException: GSS initiate failed at com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:194) at com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:272) at com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) ... 29 more appreciate, if anyone can help me.
... View more
Labels:
01-12-2021
03:25 AM
Hi We are deploying on premise CDP 7.3 with CDSW 1.8 over RHEL 7.8 which comes with python 2.6 as default. Now we are planning to upgrade python to 3.6. Is there any specific steps I need to take in cluster config beside yum installation. regards
... View more
Labels:
- Labels:
-
Cloudera Data Platform (CDP)