Reply
New Contributor
Posts: 2
Registered: ‎06-04-2015

Unable to send messages from Kafka Producer to Kafka 1.3

[ Edited ]

I had developed a Kafka Producer that worked previously for both the Kafka 0.8.1.1 and 0.8.2.1 Apache distributions.  I was hoping to re-use this same Producer for the Cloudera Kafka 1.3 distribution but have found that the Kafka Producer hangs for a whole minute until it times out.  Has anyone run into this issue as of yet and knows any kind of workaround?  It appears that Cloudera Kafka 1.3 is based on an Apache Kafka 0.8.2.0 release.

 

value.serializer = class org.apache.kafka.common.serialization.StringSerializer
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
block.on.buffer.full = true
retry.backoff.ms = 100
buffer.memory = 33554432
batch.size = 16384
metrics.sample.window.ms = 30000
metadata.max.age.ms = 300000
receive.buffer.bytes = 32768
timeout.ms = 30000
max.in.flight.requests.per.connection = 5
metric.reporters = []
bootstrap.servers = [<cloudera-kafka-server-ip>:9092]
client.id = 
compression.type = none
retries = 0
max.request.size = 1048576
send.buffer.bytes = 131072
acks = 0
reconnect.backoff.ms = 10
linger.ms = 0
metrics.num.samples = 2
metadata.fetch.timeout.ms = 60000

 

o.apache.kafka.common.network.Selector   : Error in I/O with /<cloudera-kafka-server-ip>

 

java.net.ConnectException: Connection timed out: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
at java.lang.Thread.run(Thread.java:722)

Cloudera Employee
Posts: 21
Registered: ‎07-08-2013

Re: Unable to send messages from Kafka Producer to Kafka 1.3

There isn't anything that I can think of that would have changed specific to our version that should be an issue.

 

This seems more likely to be configuration related. Can you verify that your brokers are running on 9092 and are reachable etc? 

New Contributor
Posts: 2
Registered: ‎06-04-2015

Re: Unable to send messages from Kafka Producer to Kafka 1.3

Yes they were reachable via the Cloudera command line tools when I published from another node in the cluster.  I noticed a ConnectException when running the console-producer but it was still able to publish messages to the topic.  What configuration parameters should I be looking at?