Why Kafka operations do not honor Ranger policies for users.

[#1]

SYMPTOMS : Errors such as "KeeperErrorCode = NoAuth for /config/topics"

ROOT CAUSE : Errors such as above are reported while trying to create or delete topic from an ordinary user because only the process owner of Kafka service such as root can write to zookeeper znodes i.e. /configs/topics.Ranger policies do not get enforced when a non privileged user creates a topic is because kafka-topics.sh script talks directly to zookeeper in order to create the topics. It will add entries into the zookeeper nodes and the watchers on the broker side will monitor and create topics accordingly. Due to this process involving zookeeper, the authorization cannot be done through the ranger plugin.

NEXT STEPS : If one would want to allow users to be able to create topics, We have a script called kafka-acls.sh which would help allow or deny users on topics and many such options. The details are elaborated in the document mentioned below :-

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_secure-kafka-ambari/content/ch_secure-kaf...