@Raffaele S
Yes, there is a way you will need to do a couple of thing like setting ACL's and editing Kafka and zookeeper config files to achieve that. By default the znode open permissions so anyone can change or delete a topic or znode
[zk: localhost:2181(CONNECTED) 0] getAcl /zookeeper
'world,'anyone
: cdrwa
[zk: localhost:2181(CONNECTED) 1]
Once you enable ACL's they will ONLY work on newly created topics so you restrict to r,w etc
You can achieve that but hardening zookeeper
Edit the ~/kafka/config/zookeeper.properties
# Add these 2 properties specify the authentication provider to be able to do Kerberos authentication
# Time to specify a time to take to renew the Kerberos ticket in milliseconds
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
jaasLoginRenew=3600000
Create a file zookeeper jaas file ~/kafka/config/zookeeper_jaas.conf
Server {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/path/to/zookeeper.service.keytab"
storeKey=true
useTicketCache=false
serviceName="kafka" principal="zookeeper/FQDN@REALM";
}; Edit the kafka_jaas file add a client config for zookeeper
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true keyTab="/path/to/zookeeper.service.keytab"
storeKey=true
useTicketCache=false
serviceName="kafka" principal="zookeepera/FQDN@REALM"; }; Add the below variable in your zookeeper startup scripts
Environment="KAFKA_OPTS=Djava.security.auth.login.config=/path/to/your/zookeeper_jaas.conf"
Now after modification of zookeeper and Kafka config file restart both zookeeper and kafka services Validate that the start
$ journalctl -u zookeeper | grep authenticated
The output should have INFO successfully authenticated client authenticationID=zookeeper.......
$ journalctl -u kafka | grep saslauthenticated
The output should have SaslAuthenticated Now create a new znode to test in zookeeper
Create /test-znode ""saslenabled" sasl:zookeeper/FQDN@REALM:cdwra
Now try running the getAcl without Kerberos Authentication
getAcl /test-znode 'sasl,'zookeeper/FQDN@REALM :cdrwa get /test-znode
Authenitication is not valid : /test-znode
Open a second CLI with a Kerberos authentication by executing
export KAFKA_OPTS=Djava.security.auth.login.config={/path/to/your/zookeeper_jaas.conf}
getACL /test-znode 'sasl,'zookeeper/FQDN@REALM :cdrwa
It's successful
Now go to the zookeeper kafka znode
ls /config/topics
[oracle-import,first-topic, __consumer_offsets]
getAcl /config/topics/first_topic 'world,'anyone :cdrwa
This means anyone can read, write and even delete. It would be a tedious work to add all principals of all the Kafka brokers in a cluster, to avoid limiting access to only one host in the case of a Kafka cluster like in the above case you need to edit the Kafka principal
Connect to the Kafka host edit the kafka.properties and add a propertyzookeeper.set.acl=true
Restart kafka after the above change this will force ONLY the new topics to have ACL to Kerberos principals used to connect to zookeeper
Now edit the Kafka zookeeper.properties file and add 2 properties to substitute host and REALM part of the Kerberos principal
kerberos.removeHostFromPrincipal=true
kerberos.removeRealmFromPrincipal=true
Restart Kafka/zookeeper this will now force ONLY the new topics to have ACL to Kerberos principals used to connect to zookeeper.
Now create a new topic first set the variable
export KAFKA_OPTS=Djava.security.auth.login.config={/path/to/your/zookeeper_jaas.conf} Create a new topic new-secure-topic
./bin/kafka-topic.sh --zoookeeper fqdn:2181 --create --topic new-secure-topic --replication-factor 1 --partitions 1
{set to correct replication and partitions factor}
Now connect to zookeeper and check the permissions
./bin/zookeeper-shell.sh localhost:2181
Now check the ACL note the zookeeper principal is now shortened and stripped of hostname and REALM, now everyone has only (r) read permission 🙂
getAcl /config/topics/new-secure-topic
'world,'anyone
:r
'sasl,'zookeeper
:cdrwa
Compare with the former topic created
getAcl /config/topics/first_topic
'world,'anyone
:cdrwa
Now you can restrict all the permissions cdrwa on the kafka znode !! Note topics created before enabling ACL's will not be affected
