Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Streamline App unable to access Kafka topic

Highlighted

Streamline App unable to access Kafka topic

Contributor

I'm successfully writing events on NiFi on a Kafka topic called syslog_event_topic. I've created a corresponding schema on the schema registry. I'm able to read events from that topic on the console.

[kafka@sandbox-hdf bin]$ ./kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
raw-truck_events_avro
raw-truck_speed_events_avro
syslog_event_topic
syslog_pam_events
truck_events_avro
truck_speed_events_avro

Streamline is able to see the kafka topic in the kafka source operator, and retrieve its schema. However, when I deploy and start the app, no tuples are being processed.

When I open the Storm UI from Streamline, I'm getting the following error message on the Kafka Spout:

Unable to get offset lags for kafka. Reason: org.apache.kafka.shaded.common.errors.InvalidTopicException: Topic '[syslog_event_topic]' is invalid

It's as if Storm is not able to see the topic. I've tried recycling SAM, SR, Storm to no effect. Any idea on what's causing this issue?

4 REPLIES 4

Re: Streamline App unable to access Kafka topic

Super Guru

Are there any other errors?

Highlighted

Re: Streamline App unable to access Kafka topic

Contributor

It looks like this is a fluke one. I thought this was causing Storm not being able to consume from Kafka, but after fixing another issue, this error message was still there yet Kafka messages were consumed by the Storm app.

Highlighted

Re: Streamline App unable to access Kafka topic

Super Guru

Is it there after restart?

You may need to flush out bad Kafka messages that were encoded incorrectly and restart Kafka and SAM

Highlighted

Re: Streamline App unable to access Kafka topic

Contributor

Yes, it was always there.

Don't have an account?
Coming from Hortonworks? Activate your account here