Support Questions

Find answers, ask questions, and share your expertise

Producing to Kafka HDP Sandbox 2.6.4

New Contributor

I am unable to produce message to kafka.

Using the HDP Sandbox 2.6.4 for virtual box (on mac).

  1. Imported fresh VM from .ova.
  2. Started fresh machine.
  3. Logged into Ambari as raj_ops.
  4. Started Kafka and took it out of Maintenance Mode.
  5. Started web shell http://localhost:4200
  6. Created a topic
  7. Started a the consumer
[root@sandbox-hdp ~]# cd /usr/hdp/current/kafka-broker
[root@sandbox-hdp kafka-broker]# bin/ --list --zookeeper localhost:2181
[root@sandbox-hdp kafka-broker]# bin/ --zookeeper localhost:2181 --topic simpletesttopic --partitions 1 --replication-factor 1 --create
Created topic "simpletesttopic".
[root@sandbox-hdp kafka-broker]# bin/ --list --zookeeper localhost:2181
[root@sandbox-hdp kafka-broker]# bin/ --bootstrap-server localhost:6667 --from-beginning --topic simpletesttopic

Then I opened a new web shell and used the console produce to attempt to send a message.

[root@sandbox-hdp ~]# cd /usr/hdp/current/kafka-broker
[root@sandbox-hdp kafka-broker]# bin/ --topic simpletesttopic --broker-list localhost:6667
A Test Message
[2018-07-10 17:31:47,435] ERROR Error when sending message to topic simpletesttopic with key: null, value: 14 bytes with error: (org.apache.kaf
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

So, why doesn't this work? I didn't change anything in the sandbox besides starting Kafka. Someone will come on and tell me to check the logs, OK I'll do that... there is nothing after the topic creation. Here is the last 11 lines of the controller.log and last 4 lines of the server.log.

[root@sandbox-hdp ~]# cd /usr/hdp/current/kafka-broker/logs
[root@sandbox-hdp logs]# tail -11 controller.log
[2018-07-10 17:29:25,086] DEBUG [TopicChangeListener on Controller 1001]: Topic change listener fired for path /brokers/topics with children ATLAS_HOOK,simpletesttopic,ATLAS_ENTITIES,__consumer_offsets (kafka.controller.PartitionStateMachine$TopicChangeListener)
[2018-07-10 17:29:25,089] INFO [TopicChangeListener on Controller 1001]: New topics: [Set(simpletesttopic)], deleted topics: [Set()], new partition replica assignment [Map([simpletesttopic,0] -> List(1001))] (kafka.controller.PartitionStateMachine$TopicChangeListener)
[2018-07-10 17:29:25,089] INFO [Controller 1001]: New topic creation callback for [simpletesttopic,0] (kafka.controller.KafkaController)
[2018-07-10 17:29:25,092] INFO [Controller 1001]: New partition creation callback for [simpletesttopic,0] (kafka.controller.KafkaController)
[2018-07-10 17:29:25,092] INFO [Partition state machine on Controller 1001]: Invoking state change to NewPartition for partitions [simpletesttopic,0] (kafka.controller.PartitionStateMachine)
[2018-07-10 17:29:25,096] INFO [Replica state machine on controller 1001]: Invoking state change to NewReplica for replicas [Topic=simpletesttopic,Partition=0,Replica=1001] (kafka.controller.ReplicaStateMachine)
[2018-07-10 17:29:25,098] INFO [Partition state machine on Controller 1001]: Invoking state change to OnlinePartition for partitions [simpletesttopic,0] (kafka.controller.PartitionStateMachine)
[2018-07-10 17:29:25,100] DEBUG [Partition state machine on Controller 1001]: Live assigned replicas for partition [simpletesttopic,0] are: [List(1001)] (kafka.controller.PartitionStateMachine)
[2018-07-10 17:29:25,102] DEBUG [Partition state machine on Controller 1001]: Initializing leader and isr for partition [simpletesttopic,0] to(Leader:1001,ISR:1001,LeaderEpoch:0,ControllerEpoch:2) (kafka.controller.PartitionStateMachine)
[2018-07-10 17:29:25,116] INFO [Replica state machine on controller 1001]: Invoking state change to OnlineReplica for replicas [Topic=simpletesttopic,Partition=0,Replica=1001] (kafka.controller.ReplicaStateMachine)
[2018-07-10 17:29:25,118] INFO [Controller-1001-to-broker-1001-send-thread], Controller 1001 connected to (id:1001 rack: null) for sending state change requests (kafka.controller.RequestSendThread)

[root@sandbox-hdp logs]# tail -4 server.log
[2018-07-10 17:29:25,121] INFO [ReplicaFetcherManager on broker 1001] Removed fetcher for partitions simpletesttopic-0 (kafka.server.ReplicaFetcherManager)
[2018-07-10 17:29:25,123] INFO Completed load of log simpletesttopic-0 with 1 log segments and log end offset 0 in 0 ms (kafka.log.Log)
[2018-07-10 17:29:25,124] INFO Created log for partition [simpletesttopic,0] in /kafka-logs with properties {compression.type -> producer, message.format.version -> 0.10.1-IV2, -> 60000, max.message.bytes -> 1000000, -> 0, message.timestamp.type -> CreateTime, min.insync.replicas -> 1, -> 0, preallocate -> false, min.cleanable.dirty.ratio -> 0.5, index.interval.bytes -> 4096, unclean.leader.election.enable -> true, retention.bytes -> -1, -> 86400000, cleanup.policy -> [delete], -> 9223372036854775807, -> 604800000, segment.bytes -> 1073741824, -> 604800000, -> 9223372036854775807, segment.index.bytes -> 10485760, flush.messages -> 9223372036854775807}. (kafka.log.LogManager)
[2018-07-10 17:29:25,125] INFO Partition [simpletesttopic,0] on broker 1001: No checkpointed highwatermark is found for partition [simpletesttopic,0] (kafka.cluster.Partition)

New Contributor

I am also facing same issue

Rising Star

@Eric RichardsonHi,

Could you please share the port set in Listeners property in Kafka's file. Also I would recommend to use FQDN of broker instead of localhost in producer/consumer command.

Thank you!

Hey @Eric Richardson!
Try to run it with the following command:

bin/ --topic simpletesttopic --broker-list

If does it work, change the LISTENER property inside the under the KAFKA_HOME (or through Ambari). Doing what @dbains said it's a good practice! 🙂

If it doesn't, let's figured out through the following commands:

#Let's find who's the leader
bin/ --list --zookeeper localhost:2181 --describe --topic simpletesttopic #Check on ZK namespace as well bin/zkCli or /bin/zookeeper-client ls /brokers/topics/simpletesttopic/partitions/0
If nothing the things listed above works, let's try to add the --sync to your producer.
bin/ --topic simpletesttopic --broker-list --sync

PS: I didn't test the last solution, but I looked at the code.

Hope this helps!

@Eric Richardson

Try to use your ip address or hostname instead of localhost, you can check if the port 6667 is associated to your IP or using the command below.

Netstat -atnp | grep 6667

New Contributor

It's worked for me after changing my listener ip from localhost to exact ip of virtualbox.