Hey @Jason Sphar.
Hmm, at least it give us a clue.
AFAIK the old versions of kafka used to accept zookeeper as a way to access the kafka cluster. And now you have this boostrap way (by passing any broker to reach the leader of partition + zk).
For example, i did a manual test in kafka, by creating a kafka topic, feeding with kafka-console-producer and getting the kafka-consumer-group, with the bootstrap-server parameter. I'm kinda confused now, cause in my humble opinion its seems that Nifi Processor has created the consumer-group on kafka, but kafka is unable to use..
[root@node1 ~]# kafka-topics.sh --zookeeper $ZKENSEMBLE --create --topic vin-hcc-nifi --partitions 3 --replication-factor 3
Created topic "vin-hcc-nifi".
[root@node1 ~]# kafka-console-producer.sh --broker-list node1:6667 --topic vin-hcc-nifi
>Im brazilian and im testing this topic
>please work!
>HCC
[root@node2 ~]# kafka-console-consumer.sh --bootstrap-server node2:6667 --topic vin-hcc-nifi --from-beginning
Im brazilian and im testing this topic
please work!
HCC[root@node2 ~]# kafka-consumer-groups.sh --bootstrap-server node2:6667 --list
Note: This will not show information about old Zookeeper-based consumers.
console-consumer-14185
console-consumer-81664
[root@node2 ~]# kafka-consumer-groups.sh --bootstrap-server node2:6667 --describe --group console-consumer-81664
Note: This will not show information about old Zookeeper-based consumers.
Consumer group 'console-consumer-81664' has no active members.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
vin-hcc-nifi 0 1 1 0 - - -
vin-hcc-nifi 2 1 1 0 - - -
vin-hcc-nifi 1 1 1 0 - - -
Well, anyway I saw some interesting points about the consumer -group.
========================================================
Sometimes it's useful to see the position of your consumers. We have a tool that will show the position of all consumers in a consumer group as well as how far behind the end of the log they are. To run this tool on a consumer group named my-group consuming a topic named my-topic would look like this:
| 1 2 3 4 5 6 7 8 | > bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-groupNote: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-IDmy-topic 0 2 4 2 consumer-1-029af89c-873c-4751-a720-cefd41a669d6 /127.0.0.1 consumer-1my-topic 1 2 3 1 consumer-1-029af89c-873c-4751-a720-cefd41a669d6 /127.0.0.1 consumer-1my-topic 2 2 3 1 consumer-2-42c1abd4-e3b2-425d-a8bb-e1ea49b29bb2 /127.0.0.1 consumer-2 |
This tool also works with ZooKeeper-based consumers:
| 1 2 3 4 5 6 7 8 | > bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group my-groupNote: This will only show information about consumers that use ZooKeeper (not those using the Java consumer API).TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-IDmy-topic 0 2 4 2 my-group_consumer-1my-topic 1 2 3 1 my-group_consumer-1 my-topic 2 2 3 1 my-group_consumer-2 |
========================================================
Link: https://kafka.apache.org/10/documentation.html
So after that, I made a research and fount this KB Article made by @mrodriguez (link below), we can try this to solve your problem 🙂
https://community.hortonworks.com/content/supportkb/175137/error-the-group-coordinator-is-not-availa...
Hope this helps! And a special thanks for @mrodriguez
Cheers
