Support Questions
Find answers, ask questions, and share your expertise

Kafka code stops at producer.send

Kafka code stops at producer.send

I'm following a tutorial here from MapR in making a basic Kafka producer/consumer:

I tried it both on HDP Sandbox VM 2.4 (Kafka v0.9.2) and HDP 2.5 on single-node Azure VM (Kafka v0.10.0.2).

Here is my Producer code:

package com.whatever.kafka.basic;


import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.ProducerConfig;

public class BasicProducer {

    public static void main(String[] args) throws IOException {

        /* comment-out snippet from MapR example; hard-code properties instead
        KafkaProducer<String, String> producer;
        try (InputStream props = Resources.getResource("producer.props").openStream()) {
            Properties properties = new Properties();
            producer = new KafkaProducer<String, String>(properties);

        Properties props = new Properties();
	// props.put("bootstrap.servers", "<azure-vm-fqdn>:6667"); // for HDP 2.5 Azure VM
        props.put("bootstrap.servers", ""); // for HDP 2.4 Sandbox VM
        props.put("acks", "all");
        props.put("retries", 0);
        props.put("batch.size", 16384);
        props.put("", 1000);
        props.put("", 0);
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("block.on.buffer.full", true);

        final KafkaProducer<String, String> producer = new KafkaProducer<String, String>(props);

        try {
            for (int i = 0; i < 1000000; i++) {

                System.out.println("Test 1"); // prints to console and stops on the next line

                producer.send(new ProducerRecord<String, String>(
                    "fast-messages", String.format("{\"type\":\"marker\", \"t\":%.3f, \"k\":%d}", System.nanoTime() * 1e-9, i)  

                System.out.println("Test 2");

                if (i % 250 == 0) {
                    producer.send(new ProducerRecord<String, String>(
                            "fast-messages", String.format("{\"type\":\"marker\", \"t\":%.3f, \"k\":%d}", System.nanoTime() * 1e-9, i)
                    producer.send(new ProducerRecord<String, String>(
                            "summary-markers", String.format("{\"type\":\"other\", \"t\":%.3f, \"k\":%d}", System.nanoTime() * 1e-9, i)
                    System.out.println("Sent msg number " + i);
        } catch (Throwable throwable) {
            System.out.printf("%s", throwable.getStackTrace());
        } finally {

Zookeeper and Kafka has already been started and created topics as indicated in the instruction:

$ bin/ --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic fast-messages
$ bin/ --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic summary-markers

but when I run the compiled application

$ ./kafka-example producer

I get this output (on both Sandbox and Azure VM):

log4j:WARN No appenders could be found for logger (org.apache.kafka.clients.producer.ProducerConfig)
log4j:WARN Please initialize the log4j system properly
Test 1

It gets stuck on the first producer.send line, after the Test 1 sysout log. No exception gets thrown either, it just stops at printing out that Test 1 line. When I use Eclipse to debug it using line-breaks, it stops on that line as well, apparently with nothing happening. I also tried substituting localhost, and even the external IP address of the node for the bootstrap.server properties.

The fast-messages and summary-markers directories in Kafka logs get created, but nothing written.

Any idea?