question Re: Error in NiFi Flow: in Support Questions

Error in NiFi Flow:

ssanku — Tue, 25 Oct 2016 02:44:09 GMT

Hi,

I have an error while trying to stream the data using NiFi flow, in a kerberized environment with LDAP integrated.

The error "failed while waiting for acks from Kafka" , I attached the error and properties screenshot. By the way there is a conf called " Kerberos Service Name" is that the error,

Any help is highly appreciated.

Thanks,

Sujitha

screen-shot-2016-10-24-at-124017-pm.png

screen-shot-2016-10-24-at-124006-pm.png

Re: Error in NiFi Flow:

bbende — Tue, 25 Oct 2016 03:02:27 GMT

Are you saying that Kafka is kerberized?

Currently you have the Security Protocol set to PLAINTEXT which means an unsecure Kafka.

Re: Error in NiFi Flow:

ssanku — Tue, 25 Oct 2016 07:35:58 GMT

Hi @Bryan Bende,

Thanks for the reply.

Yes I realized the error and I followed these steps,

https://community.hortonworks.com/articles/26551/accessing-kerberos-enabled-kafka-topics-using-getk.html

Also I name my principle as "nifi/iotdemo.field.hortonworks.com@LAKE"

also do I need to mention these lines in my zookeeper.properties?

3. Added 3 additional properties to the bottom of the zookeeper.properties file you have configured per the linked procedure above:

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

jaasLoginRenew=3600000

requireClientAuthScheme=sasl

Right now my error is:

"Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user"

Please find attached my PutKafka processor configurations.

Any help is highly appreciated..

screen-shot-2016-10-24-at-53412-pm.png

screen-shot-2016-10-24-at-53535-pm.png

Thanks a lot,

Sujitha

Re: Error in NiFi Flow:

bbende — Tue, 25 Oct 2016 08:46:29 GMT

Here is what you would need to do to configure PublishKafka to talk to a kerberized Kafka...

1) You can either rely on /etc/krb5.conf or you can tell NiFi to use a specific krb5.conf by setting nifi.kerberos.krb5.file= in nifi.properties to point to some other krb5.conf file.

2) Create a JAAS file lets say kafka-jaas.conf with the following:

KafkaClient {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  storeKey=true
  keyTab="/path/to/nifi-iotdemot.keytab"
  serviceName="kafka"
  principal="nifi/iotdemo.field.hortonworks.com@LAKE";
};

Changing the keyTab path to the appropriate path.

3) Specify this in NiFi's bootstrap.conf:

java.arg.15=-Djava.security.auth.login.config=/path/to/kafka-jaas.conf

4) Configure PublishKafka:

Security Protocol = PLAINTEXTSASL
Service Name = kafka

The service name should match what is in the JAAS file above.

You don't need to do any of the other stuff related to ZooKeeper, unless your NiFi instance is also using a Kerberized ZooKeeper for NiFi's state management. The above steps are what you need to do only for NiFi talking to Kafka.

Re: Error in NiFi Flow:

shalini6crore — Mon, 28 Nov 2016 17:25:45 GMT

Hi Bryan,

I follow the exact steps as mentioned by you but to no avail. I still get the ''kafka.pubsub.PublishKafka_0_10 PublishKafka_0_10[id=91c7ae98-0158-1000-aaa9-945b9c5c9f50] Failed while waiting for acks from Kafka" error. My nifi instance is standalone but configured to be secured. I am also wondering, how nifi will will create the key needed for connection to secured kafka?

Thanks for any help 🙂

Regards,

Shalini

Re: Error in NiFi Flow:

bbende — Mon, 28 Nov 2016 23:35:01 GMT

Can you start a new post describing your problem? Thanks.