Ranger policy malfunction in kafka

[#1]

In kafka, I tried to execute consume/publish command with disabled all policies of Ranger, it did not deny both consume/publish behavior. Did I miss any configuration setting of kafka or misunderstanding something else?

[#2]

@Benson Shih See this

This Kafka feature is available in HDP releases 2.3.4 (Dal-M20) or later.

[#3]

very thanks for @Neeraj Sabharwal `s response in advance,

1. I have a little confused about how to setting the ip address ranger, am I suppose to specify namenode host ip?

2. What about earlier version of HDP like 2.3.0?

[#4]

@Benson Shih You would allow the traffic from those IP.

HDP 2.3.4 ...No HDP2.3.0 or HDP 2.3.2

[#5]

I used HDP2.3.4 with ip address ranger:

after set up the policy,then I went to 140.92.27.89 command line to change user to kafka, executing Publish and Consume actions,but it still did not deny..,

[#6]

@Benson Shih, what does the Audit say? It should have the policyId which gave permission.

[#7]

Hi @bdurai,

I did not observe any information about kafka in Audit(Access); However, after I add a property "authorizer.class.name=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer" in Custom kafka-broker and executed Publish action,Audit Access could appeared information as below(why it did not show "publish" Access Type):

In addition, it get some error when I executed Publish action:

[#8]

@Benson Shih It's failing on authorization.

[#9]

Hi @Neeraj Sabharwal, it`s right that failing on authorization, but it is suppose to be authorized by Ranger right? it`s so weird that Ranger cannot control the Publish or Consume actions.

[#10]

@Benson Shih Yes..it suppose to be ..Could you do me a favor ? Please disable the kafka policy and try to see if you can run the job.

[#11]

@Neeraj Sabharwal Sure,it`s still could run the job when I disable kafka policy.

[#12]

@Benson Shih then I guess Ranger policy is working 🙂

[#13]

@Benson Shih is this issue resolved? Can you accept best answer or provide your own solution?

[#14]

Hi @Artem Ervits,

It`s still not resolved yet,I`m trying to figure out the solution soon,once I find out the solution I`ll provide solution or accept best answer.

[#15]

@Benson Shih I will be working on the demo. Let's connect...Add me on linkedin plz

[#16]

@Neeraj Sabharwa

OK ,thanks.

[#17]

Hi @Neeraj Sabharwal I would also be very interested in seeing the use case demo for this, thanks!

[#18 · Accepted solution]

Here are some steps of enable ranger for kafka and works fine with HDP2.3.4 and Ranger 0.5.0:

1.) Enable kerberos server for cluster.

2.) In Ambari server, go to Kafka`s Configs > Advanced ranger-kafka-plugin-properties , click "Enable Ranger for Kafka".

3.) Go to Configs > Custom kafka-broker , change value of "authorizer.class.name" to "org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer".

4.) Save changes and restart kafka component.

5.) Go to Ranger admin UI, then disable all policies of kafka.

6.) It should be deny Publish/Consume actions now.

[#19]

@Benson Shih This works BUT I was under the impression that you are looking for solution for non kerberos environment.

[#20]

@Benson Shih @bdurai

Benson, Take a look on this

You don't have to have kerberos to control Kafka authorization.

HDP 2.3.4

[#21]

I am having similar issue

We have non Kerberiozed Hadoop Kafka environment . I am testing integrating Ranger Kafak to secure the environment.

HDP Version: HDP-2.3.4.0-3485

This is what I did.

-- Enables Kafka plugin in Ranger.

-- Restarted Ranger

-- Create following policies in Ranger ( see the image ) ( Important : Added group Public left policy condition blank )

-- Logged in to server 21 to Produce and consume message's

-- I was able to produce and consume messages from any server .

What we want is to secure our Kafka environment through ranger by ip address. I understand that the identity of client user over a non-secure channel is not possible.

I followed the following article to secure or Kafka environment.

https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin#KafkaPlugin-Whydowehavetospecifypubl...

Please let me know what I am missing.