NiFi - PutKafka - Failed while waiting for acks from Kafka

[#1]

Trying to pass JSON Cassandra data into Kafka

Continually getting this error: Failed while waiting for acks from Kafka

Attached in the photo are my current settings.

I am confident in all of my settings except for Client Name. What is the purpose of that settings and is there a default for it?

[#2]

@jacob oatman

What's your Kafka version?

From the properties I can see that you are using PutKafka which is legacy and used for Kafka 0.8.

If you are using Kafka 0.9/10/11 use PublishKafkaXXX instead

[#3]

@Abdelkrim Hadjidj I am now getting this error: Failed to update metadata after 5000 ms.

[#4]

This is a timeout error. Which port are you using with PublishKafka_0_10? can you show your processor's configuration?

[#5]

@Abdelkrim Hadjidj I am using 2181 since that is the zookeeper port

[#6]

@Abdelkrim Hadjidj I was using putKafka instead of PublishKafka_0_10

[#7]

@jacob oatman

You should use port 6667 instead of 2181. The new Kafka API doesn't use Zookeeper directly.

Can you test with this port and let me know ?

[#8]

@Abdelkrim Hadjidj Changed the port to 6667 after the FQDN and got the same error: .TimeoutException: Failed to update metadata after 5000 ms.: Failed to update metadata after 5000 ms.

[#9]

Have you installed Kafka with Ambari? if no, maybe the port is different in your case. Check in your configuration files. Also I ca see that set the Kerberos Service Name to Kafka. Is the Kafka Kerberised? if no, why you set the Kerberos Service Name?

[#10]

@Abdelkrim Hadjidj Kafka was installed with Amabari. I did not do the installing though. The service name was a possible fix I had seen, but it has since been removed.