Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Solved Go to solution

Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Rising Star

Hello,

I am trying to send messages to a kerberos ebabled kafka using a standalone NiFi node version nifi-1.1.0.2.1.2.0-10. I followed the instructions listed in the following link: How to configure HDF1.2 to send to and get data from kerberized kafka

I inputted the keytab and principal of the kerberized kafka and added the path with the appropriate added line in the bootstrap.conf file of my NiFi instance. However whenever I try sending a message with the PutKafka processor it gives an error saying: "Failed while waiting for acks from kafka". I verified the logs in the NiFi instance and this is what I see:

2017-06-30 15:41:07,166 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null
	at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
	at java.lang.Thread.run(Thread.java:745)
2017-06-30 15:41:07,298 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress>
java.io.EOFException: null

Does anyone have any ideas of what could be going on? Any insights in this would be really helpful!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Rising Star

I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.

View solution in original post

5 REPLIES 5
Highlighted

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Contributor

@Adda Fuentes What is the kafka version on the other end? Please add a screenshot of your configuration of PutKafka, Jaas.conf file and the entry you have added in bootstrap.conf

Highlighted

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Rising Star

Hello @Shashank Chandhok,

I named the jass.conf file zookeeper-jaas.conf and the configuration is the following:

17646-zookeeperjassfile.jpg

The PutKafka configuration is the following one:

17647-putkafka.jpg

The boostrap.conf line I added is the following one:

17648-boostrapconf-file.jpg

The kafka version that we are trying to connect to is: 0.10.0

Highlighted

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Rising Star
@Shashank Chandhok

I did some changes to the jass.conf file but I am still getting the same error. This is the new jass.conf file:

17649-zookeeper-jaas-file.jpg

Highlighted

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Rising Star

I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.

View solution in original post

Highlighted

Re: Issues with sending messages to kerberos enabled kafka using standalone NiFi node version nifi-1.1.0.2.1.2.0-10 and PutKafka processor

Contributor

@Adda FuentesAwesome. Good to hear. You can mark your answer as "Accepted" so that if someone faces this issue in future, the can try and debug around the same lines.

Don't have an account?
Coming from Hortonworks? Activate your account here