Support Questions
Find answers, ask questions, and share your expertise

Kafka Producer not producing any messages

Kafka Producer not producing any messages

Super Collaborator

Hi Guys,

I have installed a new HDP cluster(2.6.0) on Google cloud platform. It has Kafka version 0.10.1. I was following this MapR link to try out a simple Kafka example. The consumer and producer properties are mentioned as below:

listeners=PLAINTEXT://instance-4.c.metal-calculus-172902.internal:6667
group.id=test
enable.auto.commit=true
key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.apache.kafka.common.serialization.StringDeserializer

# fast session timeout makes it more fun to play with failover
session.timeout.ms=10000
# These buffer sizes seem to be needed to avoid consumer switching to
# a mode where it processes one bufferful every 5 seconds with multiple
# timeouts along the way.  No idea why this happens.
fetch.min.bytes=50000
receive.buffer.bytes=262144
max.partition.fetch.bytes=2097152
listeners=PLAINTEXT://instance-4.c.metal-calculus-172902.internal:6667
acks=all
retries=0
batch.size=16384
auto.commit.interval.ms=1000
linger.ms=0
key.serializer=org.apache.kafka.common.serialization.StringSerializer
value.serializer=org.apache.kafka.common.serialization.StringSerializer
block.on.buffer.full=true

Are these properties enough? I have tested the following test topic and it worked fine.

./kafka-console-producer.sh --broker-list instance-4:6667 --topic test 
./kafka-console-consumer.sh --zookeeper instance-1:2181 --topic test --from-beginning
Don't have an account?