question Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor in Archives of Support Questions (Read Only)

Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Adda_Fuentes2 — Sat, 01 Jul 2017 05:00:48 GMT

Hello,

I am trying to send messages to a kerberos ebabled kafka using a standalone NiFi node version nifi-1.1.0.2.1.2.0-10. I followed the instructions listed in the following link: How to configure HDF1.2 to send to and get data from kerberized kafka

I inputted the keytab and principal of the kerberized kafka and added the path with the appropriate added line in the bootstrap.conf file of my NiFi instance. However whenever I try sending a message with the PutKafka processor it gives an error saying: "Failed while waiting for acks from kafka". I verified the logs in the NiFi instance and this is what I see:

2017-06-30 15:41:07,166 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null
	at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
	at java.lang.Thread.run(Thread.java:745)
2017-06-30 15:41:07,298 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null

Does anyone have any ideas of what could be going on? Any insights in this would be really helpful!

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Schandhok — Mon, 03 Jul 2017 12:14:15 GMT

@Adda Fuentes What is the kafka version on the other end? Please add a screenshot of your configuration of PutKafka, Jaas.conf file and the entry you have added in bootstrap.conf

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Adda_Fuentes2 — Sun, 18 Aug 2019 00:55:15 GMT

Hello @Shashank Chandhok,

I named the jass.conf file zookeeper-jaas.conf and the configuration is the following:

The PutKafka configuration is the following one:

The boostrap.conf line I added is the following one:

The kafka version that we are trying to connect to is: 0.10.0

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Adda_Fuentes2 — Sun, 18 Aug 2019 00:54:57 GMT

@Shashank Chandhok

I did some changes to the jass.conf file but I am still getting the same error. This is the new jass.conf file:

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Adda_Fuentes2 — Tue, 04 Jul 2017 04:12:23 GMT

I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Schandhok — Tue, 04 Jul 2017 12:16:20 GMT

@Adda FuentesAwesome. Good to hear. You can mark your answer as "Accepted" so that if someone faces this issue in future, the can try and debug around the same lines.