Ranger and Kafka integration - FAQ

[#1]

Original Article

Can I authorize access to Kafka over a non-secure channel via Ranger?

Yes. you can control access by ip-address.

Can I authorize access to Kafka over non-secure channel by user/user-groups?

No, one can’t use user/group based access to authorize Kafka access over a non-secure channel. This is because it isn't possible to assert client’s identity over the non-secure channel.

Why do we have to specify public user group on all policies items created for authorizing Kafka access over non-secure channel?

• Kafka can’t assert the identity of client user over a non-secure channel. Thus, Kafka treats all users for such access as an anonymous user (a special user literally named ANONYMOUS).
• Ranger's public user group is a means to model all users which, of course, includes this anonymous user (ANONYMOUS).

What are the specific things to watch out for when setting up authorization for accessing Kafka over non-secure channel?

• Make sure that all broker-ips have Kafka admin access to all topics, i.e. *.
• Make sure no publishers or consumers are running on broker nodes that need access control. Since broker ips have open access it isn’t possible to control access on those nodes.

Please take time to read the original article.

[#2]

Neeraj - I followed the original article and having some issue. I noticed that once I add the group "Public" in ranger policies without adding ip address in policy condition user are able to publish and consumer from any host.

This is what i did.

HDP Version: HDP-2.3.4.0-3485

-- Enables Kafka plugin in Ranger.

-- Restarted Ranger

-- Create following policies in Ranger ( see the image ) ( Important : Added group Public left policy condition blank )

-- Logged in to server 21 to Produce and consume message's

-- I was able to produce and consume messages from any server .

What we want is to secure our Kafka environment through ranger by ip address. I understand that the identity of client user over a non-secure channel is not possible.

I followed the following article to secure or Kafka environment.

https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin#KafkaPlugin-Whydowehavetospecifypubl...

Please let me know what I am missing.