Error in NiFi Flow:

[#1]

Hi,

I have an error while trying to stream the data using NiFi flow, in a kerberized environment with LDAP integrated.

The error "failed while waiting for acks from Kafka" , I attached the error and properties screenshot. By the way there is a conf called " Kerberos Service Name" is that the error,

Any help is highly appreciated.

Thanks,

Sujitha

screen-shot-2016-10-24-at-124017-pm.png

screen-shot-2016-10-24-at-124006-pm.png

[#2]

Are you saying that Kafka is kerberized?

Currently you have the Security Protocol set to PLAINTEXT which means an unsecure Kafka.

[#3]

Hi @Bryan Bende,

Thanks for the reply.

Yes I realized the error and I followed these steps,

https://community.hortonworks.com/articles/26551/accessing-kerberos-enabled-kafka-topics-using-getk....

Also I name my principle as "nifi/iotdemo.field.hortonworks.com@LAKE"

also do I need to mention these lines in my zookeeper.properties?

3. Added 3 additional properties to the bottom of the zookeeper.properties file you have configured per the linked procedure above:

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

jaasLoginRenew=3600000

requireClientAuthScheme=sasl

Right now my error is:

"Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user"

Please find attached my PutKafka processor configurations.

Any help is highly appreciated..

screen-shot-2016-10-24-at-53412-pm.png

screen-shot-2016-10-24-at-53535-pm.png

Thanks a lot,

Sujitha

[#4 · Accepted solution]

Here is what you would need to do to configure PublishKafka to talk to a kerberized Kafka...

1) You can either rely on /etc/krb5.conf or you can tell NiFi to use a specific krb5.conf by setting nifi.kerberos.krb5.file= in nifi.properties to point to some other krb5.conf file.

2) Create a JAAS file lets say kafka-jaas.conf with the following:

KafkaClient {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  storeKey=true
  keyTab="/path/to/nifi-iotdemot.keytab"
  serviceName="kafka"
  principal="nifi/iotdemo.field.hortonworks.com@LAKE";
};

Changing the keyTab path to the appropriate path.

3) Specify this in NiFi's bootstrap.conf:

java.arg.15=-Djava.security.auth.login.config=/path/to/kafka-jaas.conf

4) Configure PublishKafka:

Security Protocol = PLAINTEXTSASL
Service Name = kafka

The service name should match what is in the JAAS file above.

You don't need to do any of the other stuff related to ZooKeeper, unless your NiFi instance is also using a Kerberized ZooKeeper for NiFi's state management. The above steps are what you need to do only for NiFi talking to Kafka.

[#5]

Hi Bryan,

I follow the exact steps as mentioned by you but to no avail. I still get the ''kafka.pubsub.PublishKafka_0_10 PublishKafka_0_10[id=91c7ae98-0158-1000-aaa9-945b9c5c9f50] Failed while waiting for acks from Kafka" error. My nifi instance is standalone but configured to be secured. I am also wondering, how nifi will will create the key needed for connection to secured kafka?

Thanks for any help 🙂

Regards,

Shalini

[#6]

Can you start a new post describing your problem? Thanks.