I'm successfully writing events on NiFi on a Kafka topic called syslog_event_topic. I've created a corresponding schema on the schema registry. I'm able to read events from that topic on the console.
[kafka@sandbox-hdf bin]$ ./kafka-topics.sh --list --zookeeper localhost:2181 __consumer_offsets raw-truck_events_avro raw-truck_speed_events_avro syslog_event_topic syslog_pam_events truck_events_avro truck_speed_events_avro
Streamline is able to see the kafka topic in the kafka source operator, and retrieve its schema. However, when I deploy and start the app, no tuples are being processed.
When I open the Storm UI from Streamline, I'm getting the following error message on the Kafka Spout:
Unable to get offset lags for kafka. Reason: org.apache.kafka.shaded.common.errors.InvalidTopicException: Topic '[syslog_event_topic]' is invalid
It's as if Storm is not able to see the topic. I've tried recycling SAM, SR, Storm to no effect. Any idea on what's causing this issue?
It looks like this is a fluke one. I thought this was causing Storm not being able to consume from Kafka, but after fixing another issue, this error message was still there yet Kafka messages were consumed by the Storm app.
Is it there after restart?
You may need to flush out bad Kafka messages that were encoded incorrectly and restart Kafka and SAM