Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Storm kafka spout ClosedChannelException HDP 2.6.4.0 and kerberos

Storm kafka spout ClosedChannelException HDP 2.6.4.0 and kerberos

New Contributor

Getting the following exception in kafka spout.

Read partition information from: /icdStormTopology/icd/partition_0 --> null
2018-10-01 18:50:16.604 k.c.SimpleConsumer Thread-35-icdkafkaspout-executor[43 43] [INFO] Reconnect due to error:
java.nio.channels.ClosedChannelException: null
at kafka.network.BlockingChannel.send(BlockingChannel.scala:122) ~[stormjar.jar:?]
at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:101) [stormjar.jar:?]
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:99) [stormjar.jar:?]
at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:165) [stormjar.jar:?]
at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:86) [stormjar.jar:?]
at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:81) [stormjar.jar:?]
at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:71) [stormjar.jar:?]
at org.apache.storm.kafka.PartitionManager.<init>(PartitionManager.java:135) [stormjar.jar:?]
at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:110) [stormjar.jar:?]
at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:71) [stormjar.jar:?]
at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:130) [stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__9567$fn__9582$fn__9613.invoke(executor.clj:647) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
2018-10-01 18:50:16.611 o.a.s.util Thread-35-icdkafkaspout-executor[43 43] [ERROR] Async loop died!
java.lang.RuntimeException: java.nio.channels.ClosedChannelException
at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:115) ~[stormjar.jar:?]
at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:71) ~[stormjar.jar:?]
at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:130) ~[stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__9567$fn__9582$fn__9613.invoke(executor.clj:647) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:122) ~[stormjar.jar:?]
at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:114) ~[stormjar.jar:?]
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:99) ~[stormjar.jar:?]
at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:165) ~[stormjar.jar:?]
at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:86) ~[stormjar.jar:?]
at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:81) ~[stormjar.jar:?]
at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:71) ~[stormjar.jar:?]
at org.apache.storm.kafka.PartitionManager.<init>(PartitionManager.java:135) ~[stormjar.jar:?]
at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:110) ~[stormjar.jar:?]
... 6 more
2018-10-01 18:50:16.612 o.a.s.d.executor Thread-35-icdkafkaspout-executor[43 43] [ERROR]
java.lang.RuntimeException: java.nio.channels.ClosedChannelException
at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:115) ~[stormjar.jar:?]
at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:71) ~[stormjar.jar:?]
at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:130) ~[stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__9567$fn__9582$fn__9613.invoke(executor.clj:647) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]

Don't have an account?
Coming from Hortonworks? Activate your account here