Created 06-30-2017 10:00 PM
Hello,
I am trying to send messages to a kerberos ebabled kafka using a standalone NiFi node version nifi-1.1.0.2.1.2.0-10. I followed the instructions listed in the following link: How to configure HDF1.2 to send to and get data from kerberized kafka
I inputted the keytab and principal of the kerberized kafka and added the path with the appropriate added line in the bootstrap.conf file of my NiFi instance. However whenever I try sending a message with the PutKafka processor it gives an error saying: "Failed while waiting for acks from kafka". I verified the logs in the NiFi instance and this is what I see:
2017-06-30 15:41:07,166 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress> java.io.EOFException: null at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) at org.apache.kafka.common.network.Selector.poll(Selector.java:248) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) at java.lang.Thread.run(Thread.java:745) 2017-06-30 15:41:07,298 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress> java.io.EOFException: null
Does anyone have any ideas of what could be going on? Any insights in this would be really helpful!
Created 07-03-2017 09:12 PM
I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.
Created 07-03-2017 05:14 AM
@Adda Fuentes What is the kafka version on the other end? Please add a screenshot of your configuration of PutKafka, Jaas.conf file and the entry you have added in bootstrap.conf
Created on 07-03-2017 04:01 PM - edited 08-17-2019 05:55 PM
Hello @Shashank Chandhok,
I named the jass.conf file zookeeper-jaas.conf and the configuration is the following:
The PutKafka configuration is the following one:
The boostrap.conf line I added is the following one:
The kafka version that we are trying to connect to is: 0.10.0
Created on 07-03-2017 06:14 PM - edited 08-17-2019 05:54 PM
I did some changes to the jass.conf file but I am still getting the same error. This is the new jass.conf file:
Created 07-03-2017 09:12 PM
I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.
Created 07-04-2017 05:16 AM
@Adda FuentesAwesome. Good to hear. You can mark your answer as "Accepted" so that if someone faces this issue in future, the can try and debug around the same lines.