Support Questions

Find answers, ask questions, and share your expertise

Streamline App unable to access Kafka topic

Contributor

I'm successfully writing events on NiFi on a Kafka topic called syslog_event_topic. I've created a corresponding schema on the schema registry. I'm able to read events from that topic on the console.

[kafka@sandbox-hdf bin]$ ./kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
raw-truck_events_avro
raw-truck_speed_events_avro
syslog_event_topic
syslog_pam_events
truck_events_avro
truck_speed_events_avro

Streamline is able to see the kafka topic in the kafka source operator, and retrieve its schema. However, when I deploy and start the app, no tuples are being processed.

When I open the Storm UI from Streamline, I'm getting the following error message on the Kafka Spout:

Unable to get offset lags for kafka. Reason: org.apache.kafka.shaded.common.errors.InvalidTopicException: Topic '[syslog_event_topic]' is invalid

It's as if Storm is not able to see the topic. I've tried recycling SAM, SR, Storm to no effect. Any idea on what's causing this issue?

4 REPLIES 4

Super Guru

Are there any other errors?

Contributor

It looks like this is a fluke one. I thought this was causing Storm not being able to consume from Kafka, but after fixing another issue, this error message was still there yet Kafka messages were consumed by the Storm app.

Super Guru

Is it there after restart?

You may need to flush out bad Kafka messages that were encoded incorrectly and restart Kafka and SAM

Contributor

Yes, it was always there.