Created 01-17-2019 02:54 PM
Hello,
I have enable kerberos on cluster. I can successfully connect to consumer through kafka-konsole-consumer.
But when i try to connect to topic through NiFi ConsumeKafka processor i have error:
WARN [Timer-Driven Process Thread-8] o.a.n.p.kafka.pubsub.ConsumeKafka_1_0 ConsumeKafka_1_0[id=504e5811-0168-1000-0000-000024c83cc5] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@51911c24. Will roll back session and discard any partially received data. WARN [kafka-kerberos-refresh-thread-nifi/*@*] o.a.k.c.security.kerberos.KerberosLogin [Principal=nifi/*@*]: TGT renewal thread has been interrupted and will exit.
Created 01-22-2019 07:51 PM
Can you log on both cluster as zookeeper run kinit then
$ /usr/hdp/current/zookeeper-server/bin/zkCli.sh [zk: localhost:2181(CONNECTED) 0] getAcl /config/topics
Compare the outputs there should be a difference between the 2 cluster
sample output
[zk: localhost:2181(CONNECTED) 1] getAcl /config/topics 'world,'anyone : r 'sasl,'kafka : cdrwa
And on the other
[zk: localhost:2181(CONNECTED) 1] getAcl /config/topics 'world,'anyone : cdrwa
If that's the case have a look at how to set the correct ACL for the /config/topics in this HCC document
Please let me know
Created 01-23-2019 09:45 AM
Can i remove zookeeper's trees? Will they be recreate after deleting with right permission?
Created 01-23-2019 08:19 AM
Can you compare these files on both cluster, if it doesn't exist on the DEV then create them with correct owner and privileges
Kafka
/etc/kafka/conf/kafka_client_jaas.conf /etc/kafka/conf/kafka_jaas.conf
Zookeeper
/etc/zookeeper/conf/zookeeper_jaas.conf /etc/zookeeper/conf/zookeeper_client_jaas.conf
After updating these files restart the zk and Kafka and retry
HTH
Created 01-23-2019 08:44 AM
I compared files, yes their exists and the same.
Created 01-23-2019 10:07 AM
Logon to Zk on DEV and PROD Check whether you have child znodes under
ls /config/topics
Validate that you have Kerberos entries in /etc/nifi/conf/nifi_jaas.conf
example
NiFiClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/nifi.service.keytab" storeKey=true useTicketCache=false principal="nifi/{kdc_host}@REALM"; }; RegistryClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/nifi.service.keytab" storeKey=true useTicketCache=false principal="nifi/{kdc_host}@REALM";
Please revert
Created 01-23-2019 01:17 PM
Yes i have
ls /config/topics
[test1]
NiFiClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/etc/security/keytabs/nifi.service.keytab"
storeKey=true
useTicketCache=false
principal="nifi/host@RAIFFEISEN.RU";
};
RegistryClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/etc/security/keytabs/nifi.service.keytab"
storeKey=true
useTicketCache=false
principal="nifi/host@RAIFFEISEN.RU";
Created 01-23-2019 04:55 PM
I think the topics created before changing the ACL's don't inherit the permissions. From the znode entry, I see you have one topic called test.
To validate can you create a new topic and ensure you change them in your Nifi flow files and retest! Normally if you run rmr in zookeeper CLI the entry deleted should be created but I am not sure for Kafka topics.
Please revert
Created on 02-04-2019 04:11 PM - edited 08-17-2019 02:58 PM
Thank you, you are right, the problem was really in zookeeper's acl.
I copied everything in "ZooKeeper directory" from Test cluster to Dev cluster and that was help. But i don't know what exactly permission affected it. Is something way to get list all acl permission by Zookeeper? I would like to compare it with all acl from both cluster.