Member since
08-22-2017
1
Post
0
Kudos Received
0
Solutions
08-22-2017
04:24 PM
Hi,
I'm deploying the Storm topology from the following hortonworks tutorial:
https://hortonworks.com/hadoop-tutorial/processing-trucking-iot-data-with-apache-storm/
The topology deploys well into the cluster, but the kafka spout is not able to fetch any data from the kafka topic. All configuration is as specificied in the tutorial.
The cluster is kerberized, so I'm leveraging SASL_PLAINTEXT as security protocol.
As per the logs, I'm just getting a few WARNINGS:
o.a.s.c.zookeeper-state-factory main-EventThread [WARN] Received event ::none: with disconnected Writer Zookeeper. o.a.k.c.NetworkClient Thread-8-enrichedTruckData-executor[3 3] [WARN] Error while fetching metadata with correlation id 1 : {trucking_data_truck=UNKNOWN_TOPIC_OR_PARTITION}
The first warning corresponds to line 44 in the following clojure code file:
https://github.com/apache/storm/blob/1.1.x-branch/storm-core/src/clj/org/apache/storm/cluster_state/zookeeper_state_factory.clj
So there seems to be some issues between storm and zookeeper but can't seem to narrow it down, as all ACLS in zookeeper's node/paths are correct as well as the kafka ones, in terms of users and permissions.
I've actually run a test deleting all topics and I'm getting the same output so...clearly there's an issue with the spout that is not able to read the kafka topics, that creates this knock on effect by which the storm consumer node is not created in zookeeper either to store offsets.
Not sure if anyone has come up across a similar issue, any suggestion or help would be highly appreciated.
Thanks!!
... View more
Labels: