Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 09-27-2024 12:13 AM
Hey Team,
I am want to connect hive with SSL and Kerberos.
Getting error with trying to connect via beeline and got the following error:
root@ip-172-31-13-77:~# beeline -u "jdbc:hive2://ec2-52-66-58-15.ap-south-1.compute.amazonaws.com:10000/default;principal=hive/ec2-52-66-58-15.ap-south-1.compute.amazonaws.com@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=******************"
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/log4j-slf4j-impl-2.13.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/log4j-slf4j-impl-2.13.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://ec2-52-66-58-15.ap-south-1.compute.amazonaws.com:10000/default;principal=hive/ec2-52-66-58-15.ap-south-1.compute.amazonaws.com@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=W9KFVmohGT4H0wk90viVjP6NLljSUjNBSHs4gnKd4uS
24/09/27 07:11:34 [main]: WARN jdbc.HiveConnection: Failed to connect to ec2-52-66-58-15.ap-south-1.compute.amazonaws.com:10000
Unknown HS2 problem when communicating with Thrift server.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://ec2-52-66-58-15.ap-south-1.compute.amazonaws.com:10000/default;principal=hive/ec2-52-66-58-15.ap-south-1.compute.amazonaws.com@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=************************: javax.net.ssl.SSLHandshakeException: No subject alternative DNS name matching ec2-52-66-58-15.ap-south-1.compute.amazonaws.com found. (state=08S01,code=0)
Looking for help and get it fix.
It is little urgent please reply at the earliest
Thanks in advance
Ankit Bishnoi
Created 09-27-2024 12:28 AM
1)Please check for a valid kerberos ticket before connecting to hive
$Kinit
$beeline
2) Can you please check the hive metastore logs for more details
Created 09-27-2024 05:56 AM
Hey
thanks for reaching me out @hadoopranger
After kinit I am trying to connect via beeline:
Please logs of the hive metastore:
2024-09-27 12:51:13,989 INFO org.apache.hadoop.fs.TrashPolicyDefault: [pool-5-thread-62]: Moved: 'hdfs://ip-172-31-13-77.ap-south-1.compute.internal:8020/user/hue/.cloudera_manager_hive_metastore_canary/cloudera_manager_metastore_canary_test_catalog_hive_HIVEMETASTORE_ac9f056b57b5af07c85fc6a689cb47ce' to trash at: hdfs://ip-172-31-13-77.ap-south-1.compute.internal:8020/user/hue/.Trash/Current/user/hue/.cloudera_manager_hive_metastore_canary/cloudera_manager_metastore_canary_test_catalog_hive_HIVEMETASTORE_ac9f056b57b5af07c85fc6a689cb47ce1727441473987
2024-09-27 12:51:13,993 WARN org.apache.hadoop.hive.metastore.utils.FileUtils: [pool-5-thread-62]: File does not exist: hdfs://ip-172-31-13-77.ap-south-1.compute.internal:8020/user/hue/.cloudera_manager_hive_metastore_canary/cloudera_manager_metastore_canary_test_catalog_hive_HIVEMETASTORE_ac9f056b57b5af07c85fc6a689cb47ce; Force to delete it.
2024-09-27 12:51:13,994 ERROR org.apache.hadoop.hive.metastore.utils.FileUtils: [pool-5-thread-62]: Failed to delete hdfs://ip-172-31-13-77.ap-south-1.compute.internal:8020/user/hue/.cloudera_manager_hive_metastore_canary/cloudera_manager_metastore_canary_test_catalog_hive_HIVEMETASTORE_ac9f056b57b5af07c85fc6a689cb47ce
2024-09-27 12:51:13,995 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-5-thread-62]: 62: Cleaning up thread local RawStore...
2024-09-27 12:51:13,995 INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-62]: ugi=hue/ip-172-31-13-77.ap-south-1.compute.internal@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM ip=172.31.13.77 cmd=Cleaning up thread local RawStore...
2024-09-27 12:51:13,995 INFO org.apache.hadoop.hive.metastore.ObjectStore: [pool-5-thread-62]: RawStore: org.apache.hadoop.hive.metastore.ObjectStore@b6195da, with PersistenceManager: org.datanucleus.api.jdo.JDOPersistenceManager@142daedd will be shutdown
2024-09-27 12:51:13,995 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-5-thread-62]: 62: Done cleaning up thread local RawStore
2024-09-27 12:51:13,995 INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-62]: ugi=hue/ip-172-31-13-77.ap-south-1.compute.internal@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM ip=172.31.13.77 cmd=Done cleaning up thread local RawStore
2024-09-27 12:51:13,995 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-5-thread-62]: 62: Done cleaning up thread local RawStore
2024-09-27 12:51:13,995 INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-62]: ugi=hive/ip-172-31-13-77.ap-south-1.compute.internal@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM ip=172.31.13.77 cmd=Done cleaning up thread local RawStore
Output of the klist command:
root@ip-172-31-13-77:/var/log/hive# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hive/ec2-52-66-58-15.ap-south-1.compute.amazonaws.com@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM
Valid starting Expires Service principal
09/27/24 06:50:48 09/27/24 16:50:48 krbtgt/AP-SOUTH-1.COMPUTE.AMAZONAWS.COM@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM
renew until 10/04/24 06:50:39
Moreover;Also check the logs of the hiveserver also:
root@ip-172-31-13-77:/var/log/hive# tail -f hadoop-cmf-hive-HIVESERVER2-ip-172-31-13-77.ap-south-1.compute.internal.log.out
at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_232]
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
... 10 more
2024-09-27 12:31:54,081 INFO org.apache.hadoop.hive.ql.metadata.HiveMaterializedViewsRegistry: [HiveMaterializedViewsRegistry-0]: Materialized views registry has been refreshed
Please let me know if any things needs to be check and configured
Thanks
Ankit