Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

avatar
Expert Contributor

Hello,

I am trying to send messages to a kerberos ebabled kafka using a standalone NiFi node version nifi-1.1.0.2.1.2.0-10. I followed the instructions listed in the following link: How to configure HDF1.2 to send to and get data from kerberized kafka

I inputted the keytab and principal of the kerberized kafka and added the path with the appropriate added line in the bootstrap.conf file of my NiFi instance. However whenever I try sending a message with the PutKafka processor it gives an error saying: "Failed while waiting for acks from kafka". I verified the logs in the NiFi instance and this is what I see:

2017-06-30 15:41:07,166 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null
	at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
	at java.lang.Thread.run(Thread.java:745)
2017-06-30 15:41:07,298 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null

Does anyone have any ideas of what could be going on? Any insights in this would be really helpful!

1 ACCEPTED SOLUTION

avatar
Expert Contributor

I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.

View solution in original post

5 REPLIES 5

avatar
Rising Star

@Adda Fuentes What is the kafka version on the other end? Please add a screenshot of your configuration of PutKafka, Jaas.conf file and the entry you have added in bootstrap.conf

avatar
Expert Contributor

Hello @Shashank Chandhok,

I named the jass.conf file zookeeper-jaas.conf and the configuration is the following:

17646-zookeeperjassfile.jpg

The PutKafka configuration is the following one:

17647-putkafka.jpg

The boostrap.conf line I added is the following one:

17648-boostrapconf-file.jpg

The kafka version that we are trying to connect to is: 0.10.0

avatar
Expert Contributor
@Shashank Chandhok

I did some changes to the jass.conf file but I am still getting the same error. This is the new jass.conf file:

17649-zookeeper-jaas-file.jpg

avatar
Expert Contributor

I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.

avatar
Rising Star

@Adda FuentesAwesome. Good to hear. You can mark your answer as "Accepted" so that if someone faces this issue in future, the can try and debug around the same lines.