Support Questions
Find answers, ask questions, and share your expertise

Storm Spout Not Finding Kafka Topic

New Contributor

Hi,

I'm deploying the Storm topology from the following hortonworks tutorial:

The topology deploys well into the cluster, but the kafka spout is not able to fetch any data from the kafka topic. All configuration is as specificied in the tutorial.

The cluster is kerberized, so I'm leveraging SASL_PLAINTEXT as security protocol.

As per the logs, I'm just getting a few WARNINGS:

  1. o.a.s.c.zookeeper-state-factory main-EventThread [WARN] Received event ::none: with disconnected Writer Zookeeper.
  2. o.a.k.c.NetworkClient Thread-8-enrichedTruckData-executor[3 3] [WARN] Error while fetching metadata with correlation id 1 : {trucking_data_truck=UNKNOWN_TOPIC_OR_PARTITION}

The first warning corresponds to line 44 in the following clojure code file:

So there seems to be some issues between storm and zookeeper but can't seem to narrow it down, as all ACLS in zookeeper's node/paths are correct as well as the kafka ones, in terms of users and permissions.

I've actually run a test deleting all topics and I'm getting the same output so...clearly there's an issue with the spout that is not able to read the kafka topics, that creates this knock on effect by which the storm consumer node is not created in zookeeper either to store offsets.

Not sure if anyone has come up across a similar issue, any suggestion or help would be highly appreciated.

Thanks!!

1 REPLY 1

New Contributor

Have you solved this? I am faced with exactly the same issue

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.