Please find the attached screen shot.. Not able to attach full role log   ... View more

Hi @Scharan ,   I tried to remove the below PAM properties but not removing . It automatically adding . also I have added the admin user in the below property "zeppelin.shiro.user.block" . But still same issue . Please advise.       ... View more

Hi @carrossoni , I followed the same as mentioned in below . But still getting this error and Not able to find the RC . Please advise. SERVICE=[zookeeper], SERVICE_TYPE=[ZOOKEEPER], LOG_LEVEL=[WARN], SEVERITY=[IMPORTANT]}, content=Cannot open channel to 3 at election address matrix-mlsec-05.cisco.com/10.83.35.145:4181, timestamp=1599765151156} - 1 of 56 failure(s) in last 1831s java.io.IOException: Error connecting to matrix-mlsec-01.cisco.com/10.83.35.141:7184 at com.cloudera.cmf.event.shaded.org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:292) at com.cloudera.cmf.event.shaded.org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:206) at com.cloudera.cmf.event.shaded.org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:141) at com.cloudera.cmf.event.publish.AvroEventStorePublishProxy.checkSpecificRequestor(AvroEventStorePublishProxy.java:135) at com.cloudera.cmf.event.publish.AvroEventStorePublishProxy.publishEvent(AvroEventStorePublishProxy.java:210) at com.cloudera.cmf.event.publish.EventStorePublisherWithRetry$PublishEventTask.run(EventStorePublisherWithRetry.java:242) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152) at com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at com.cloudera.cmf.event.shaded.org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at com.cloudera.cmf.event.shaded.org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ... 3 more .       ... View more

Below error in Zookeeper    2020-09-18 14:30:38,331 ERROR org.apache.zookeeper.server.quorum.LearnerHandler: Unexpected exception causing shutdown while sock still open java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:197) at java.io.DataInputStream.readFully(DataInputStream.java:169) at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:113) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:87) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:118) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:379) 2020-09-18 14:30:38,332 WARN org.apache.zookeeper.server.quorum.LearnerHandler: ******* GOODBYE /64.102.14.234:55960 ******** 2020-09-18 14:30:38,337 ERROR org.apache.zookeeper.server.quorum.LearnerHandler: Unexpected exception causing shutdown while sock still open java.io.IOException: Unreasonable length = 541611092 at org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:146) at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:111) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:87) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:118) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:379) 2020-09-18 14:30:38,338 WARN org.apache.zookeeper.server.quorum.LearnerHandler: ******* GOODBYE /64.102.14.234:56947 ******** 2020-09-18 14:31:41,053 INFO org.apache.zookeeper.server.ZooKeeperServer: Submitting global closeSession request for session 0x300261365d7006e ... View more

Hi @carrossoni , You are Right . Hive server 2 is down on Hive on Tez. Please find the Error log below. Please advise how to overcome this issue. 20/09/24 02:39:04 ERROR server.HiveServer2: [shutdown-hook-0]: Error stopping schq java.lang.IllegalStateException: The current ScheduledQueryExecutionService INSTANCE is invalid at org.apache.hadoop.hive.ql.scheduled.ScheduledQueryExecutionService.close(ScheduledQueryExecutionService.java:308) ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100] at org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:886) ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100] at org.apache.hive.service.server.HiveServer2.lambda$init$0(HiveServer2.java:433) ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_232] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_232] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232] s  ... View more