I am trying to the truck iot app. Everything is setup and apparently running except that I don't see any processing in Streaming Analytics Manager. This is not surprising because when I try to check my Kafka topic using "kafka-console-consumer", I get nothing:
root@mqureshi4 bin]# ./kafka-console-consumer.sh --bootstrap-server localhost:6667 --topic truck_speed_events_avro --from-beginning
Nifi shows thousands of records have been sent. Data Provenance in Nifi shows records being "sent" by Publish Kafka Processor. What could be wrong here? I have verified and there are no typos in topic names.
Actually, I am able to see my data in Kafka. its under /kafka-logs" folder as configured in Kafka settings in Ambari. But my zookeeper directory seems almost emopty. I have only the following:
[root@mqureshi4 truck_events_avro-0]# ls -l /hadoop/zookeeper/ total 4 -rw-r--r-- 1 root root 1 Mar 16 15:50 myid drwxr-xr-x 2 zookeeper hadoop 18 Mar 16 15:50 version-2 [root@mqureshi4 truck_events_avro-0]# ls -l /hadoop/zookeeper/version-2/ total 22064 -rw-r--r-- 1 zookeeper hadoop 67108880 Mar 20 18:29 log.1
I don't see a Kafka node here. I am wondering if that's the issue.
So me see I understand:
This seems to be an issue. Perhaps case sensitive? I have this issue as well. Kafka messages not picked up by SAM.