Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Hiveserver2 on Tez can't start after enable kerberos

avatar
New Contributor
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/log4j-slf4j-impl-2.13.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-12-05 13:22:59: Starting HiveServer2
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/log4j-slf4j-impl-2.13.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = e5b3a081-be83-40f7-bbd9-63ad7c06629b
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-policymgr-ssl.xml): couldn't find resource file location
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-policymgr-ssl.xml): couldn't find resource file location
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-cm_hive-audit.xml): couldn't find resource file location
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-cm_hive-audit.xml): couldn't find resource file location
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-cm_hive-security.xml): couldn't find resource file location
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-cm_hive-security.xml): couldn't find resource file location
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-cm_hive-policymgr-ssl.xml): couldn't find resource file location
22/12/05 13:23:04 ERROR config.RangerConfiguration: [main]: addResourceIfReadable(ranger-hive-cm_hive-policymgr-ssl.xml): couldn't find resource file location
Hive Session ID = e892950e-5611-46f3-abcf-95ac24e73611
22/12/05 13:23:11 ERROR server.HiveServer2: [main]: Error starting  Tez sessions: 
org.apache.hive.service.ServiceException: Unable to setup tez session pool
	at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:825) ~[hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:795) ~[hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:718) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1059) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.access$1400(HiveServer2.java:138) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1333) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1177) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_232]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_232]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
	at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.1.7.1.7.0-551.jar:?]
	at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.1.7.1.7.0-551.jar:?]
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1670230527083_0136 failed 2 times due to AM Container for appattempt_1670230527083_0136_000002 exited with  exitCode: -1000
Failing this attempt.Diagnostics: [2022-12-05 13:23:10.965]Application application_1670230527083_0136 initialization failed (exitCode=255) with output: main : command provided 0
main : run as user is hive
main : requested yarn user is hive
Requested user hive is banned

For more detailed output, check the application tracking page: http://nn1-be:8088/cluster/app/application_1670230527083_0136 Then click on links to logs of each attempt.
. Failing the application.
	at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:979) ~[tez-api-0.9.1.7.1.7.0-551.jar:0.9.1.7.1.7.0-551]
	at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:948) ~[tez-api-0.9.1.7.1.7.0-551.jar:0.9.1.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:556) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:374) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:313) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:118) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPool.startInitialSession(TezSessionPool.java:359) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPool.startUnderInitLock(TezSessionPool.java:171) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPool.start(TezSessionPool.java:123) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:115) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:822) ~[hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	... 12 more
22/12/05 13:23:11 ERROR server.HiveServer2: [main]: Error starting  Tez sessions: 
org.apache.hive.service.ServiceException: Unable to setup tez session pool
	at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:825) ~[hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:795) ~[hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:718) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1059) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.access$1400(HiveServer2.java:138) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1333) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1177) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_232]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_232]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
	at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.1.7.1.7.0-551.jar:?]
	at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.1.7.1.7.0-551.jar:?]
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1670230527083_0136 failed 2 times due to AM Container for appattempt_1670230527083_0136_000002 exited with  exitCode: -1000
Failing this attempt.Diagnostics: [2022-12-05 13:23:10.965]Application application_1670230527083_0136 initialization failed (exitCode=255) with output: main : command provided 0
main : run as user is hive
main : requested yarn user is hive
Requested user hive is banned

For more detailed output, check the application tracking page: http://nn1-be:8088/cluster/app/application_1670230527083_0136 Then click on links to logs of each attempt.
. Failing the application.
	at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:979) ~[tez-api-0.9.1.7.1.7.0-551.jar:0.9.1.7.1.7.0-551]
	at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:948) ~[tez-api-0.9.1.7.1.7.0-551.jar:0.9.1.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:556) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:374) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:313) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.open(TezSessionPoolSession.java:118) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPool.startInitialSession(TezSessionPool.java:359) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPool.startUnderInitLock(TezSessionPool.java:171) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPool.start(TezSessionPool.java:123) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:115) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:822) ~[hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	... 12 more
22/12/05 13:23:12 ERROR server.HiveServer2: [main-EventThread]: Error stopping schq
java.lang.IllegalStateException: The current ScheduledQueryExecutionService INSTANCE is invalid
	at org.apache.hadoop.hive.ql.scheduled.ScheduledQueryExecutionService.close(ScheduledQueryExecutionService.java:312) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:892) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2$DeRegisterWatcher.process(HiveServer2.java:617) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:77) [curator-framework-4.3.0.7.1.7.0-551.jar:4.3.0.7.1.7.0-551]
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) [zookeeper-3.5.5.7.1.7.0-551.jar:3.5.5.7.1.7.0-551]
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) [zookeeper-3.5.5.7.1.7.0-551.jar:3.5.5.7.1.7.0-551]
22/12/05 13:23:12 ERROR server.HiveServer2: [main-EventThread]: Error stopping schq
java.lang.IllegalStateException: The current ScheduledQueryExecutionService INSTANCE is invalid
	at org.apache.hadoop.hive.ql.scheduled.ScheduledQueryExecutionService.close(ScheduledQueryExecutionService.java:312) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:892) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.hive.service.server.HiveServer2$DeRegisterWatcher.process(HiveServer2.java:617) [hive-service-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
	at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:77) [curator-framework-4.3.0.7.1.7.0-551.jar:4.3.0.7.1.7.0-551]
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) [zookeeper-3.5.5.7.1.7.0-551.jar:3.5.5.7.1.7.0-551]
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) [zookeeper-3.5.5.7.1.7.0-551.jar:3.5.5.7.1.7.0-551]
1 ACCEPTED SOLUTION

avatar
Guru

@Elbehwash 

 

1. Please navigate to CM ==> Hive on tez ==>  COnfiguration ==> Hiveserver2 safety valve for hive-site.xml

2. Add

 

   hive.server2.tez.initialize.default.sessions = false;

 

3. restart and check

 

View solution in original post

3 REPLIES 3

avatar
Guru

@Elbehwash 

 

1. Please navigate to CM ==> Hive on tez ==>  COnfiguration ==> Hiveserver2 safety valve for hive-site.xml

2. Add

 

   hive.server2.tez.initialize.default.sessions = false;

 

3. restart and check

 

avatar
New Contributor

Thanks a lot it is working fine 🤗

avatar
Guru

Please "Accept as solution" , if this has worked