Support Questions
Find answers, ask questions, and share your expertise

Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Rising Star

Hello,

I am trying to send messages to a kerberos ebabled kafka using a standalone NiFi node version nifi-1.1.0.2.1.2.0-10. I followed the instructions listed in the following link: How to configure HDF1.2 to send to and get data from kerberized kafka

I inputted the keytab and principal of the kerberized kafka and added the path with the appropriate added line in the bootstrap.conf file of my NiFi instance. However whenever I try sending a message with the PutKafka processor it gives an error saying: "Failed while waiting for acks from kafka". I verified the logs in the NiFi instance and this is what I see:

2017-06-30 15:41:07,166 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null
	at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
	at java.lang.Thread.run(Thread.java:745)
2017-06-30 15:41:07,298 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null

Does anyone have any ideas of what could be going on? Any insights in this would be really helpful!

1 ACCEPTED SOLUTION

Accepted Solutions

Rising Star

I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.

View solution in original post

5 REPLIES 5

Contributor

@Adda Fuentes What is the kafka version on the other end? Please add a screenshot of your configuration of PutKafka, Jaas.conf file and the entry you have added in bootstrap.conf

Rising Star

Hello @Shashank Chandhok,

I named the jass.conf file zookeeper-jaas.conf and the configuration is the following:

17646-zookeeperjassfile.jpg

The PutKafka configuration is the following one:

17647-putkafka.jpg

The boostrap.conf line I added is the following one:

17648-boostrapconf-file.jpg

The kafka version that we are trying to connect to is: 0.10.0

Rising Star
@Shashank Chandhok

I did some changes to the jass.conf file but I am still getting the same error. This is the new jass.conf file:

17649-zookeeper-jaas-file.jpg

Rising Star

I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.

View solution in original post

Contributor

@Adda FuentesAwesome. Good to hear. You can mark your answer as "Accepted" so that if someone faces this issue in future, the can try and debug around the same lines.