question Re: Ranger policy malfunction in kafka in Support Questions

Ranger policy malfunction in kafka

bensonshih — Thu, 14 Jan 2016 15:22:57 GMT

In kafka, I tried to execute consume/publish command with disabled all policies of Ranger, it did not deny both consume/publish behavior. Did I miss any configuration setting of kafka or misunderstanding something else?

Re: Ranger policy malfunction in kafka

nsabharwal — Mon, 19 Aug 2019 12:14:09 GMT

@bdurai @bganesan

I was able to reproduce this. I have only kafka user listed in Kafka policy and root can consume and produce the data "not listed in kafka policy.

@Benson Shih

Re: Ranger policy malfunction in kafka

alal1 — Fri, 15 Jan 2016 01:31:04 GMT

Is the Ranger plugin properly installed? For example, do you any evidence of it in Ranger Audit logs, e.g. kafaka server connecting to Ranger to download policies or access log indicating that access was allowed by ranger?

Re: Ranger policy malfunction in kafka

pbrahmbhatt — Fri, 15 Jan 2016 03:34:44 GMT

Please check your server.properties file and ensure you have authorizer.class.name set to Ranger Authorizer's Fully Qualified class name.

Re: Ranger policy malfunction in kafka

bdurai — Fri, 15 Jan 2016 09:12:20 GMT

Also look into the Ranger Audits from the Ranger Admin. If Ranger is allowing the request, then it will have policy which gave the permission.

Re: Ranger policy malfunction in kafka

aervits — Fri, 15 Jan 2016 18:45:21 GMT

@Benson Shih

did you turn off the global allow policy for Kafka?

Re: Ranger policy malfunction in kafka

bensonshih — Mon, 18 Jan 2016 09:34:28 GMT

I will check for it

Re: Ranger policy malfunction in kafka

bensonshih — Mon, 18 Jan 2016 09:36:28 GMT

It`s supposes to be "org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer" right?

Re: Ranger policy malfunction in kafka

bensonshih — Mon, 18 Jan 2016 09:37:36 GMT

What is it mean? Could you give me an example thanks.

Re: Ranger policy malfunction in kafka

bensonshih — Mon, 18 Jan 2016 09:38:47 GMT

I will check for it, too

Re: Ranger policy malfunction in kafka

nsabharwal — Tue, 19 Jan 2016 10:32:42 GMT

@Benson Shih check this https://github.com/abajwa-hw/security-workshops/blob/master/Setup-ranger-23.md#setup-kafka-plugin-for-ranger

Re: Ranger policy malfunction in kafka

ashaver — Thu, 28 Jan 2016 03:55:07 GMT

Is this issue resolved? I also tried to create a kafka ranger policy to exclude a select user from not creating or deleting topics. But it doesn't get enforced. I see the 200 response in Ranger Audits that Kafka plugin is up.

Re: Ranger policy malfunction in kafka

bensonshih — Sun, 31 Jan 2016 22:29:33 GMT

Is it correct that the kafka and ranger must be in the kerberized cluster environment?

Re: Ranger policy malfunction in kafka

nsabharwal — Mon, 01 Feb 2016 00:22:42 GMT

@Benson Shih See this

https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin#KafkaPlugin-CanIauthorizeraccesstoKafkaoveranon-securechannelviaRanger?

Re: Ranger policy malfunction in kafka

nsabharwal — Mon, 01 Feb 2016 00:31:37 GMT

@Benson Shih Just created an article based on this https://community.hortonworks.com/articles/12699/ranger-and-kafka-integration-faq.html

Very useful to resolve this issue.

Re: Ranger policy malfunction in kafka

nsabharwal — Mon, 01 Feb 2016 00:35:19 GMT

@Anna Shaverdian see this https://community.hortonworks.com/articles/12699/ranger-and-kafka-integration-faq.html

Re: Ranger policy malfunction in kafka

bensonshih — Mon, 19 Aug 2019 12:14:01 GMT

Hi @Neeraj Sabharwal,

I still can not deny Publish and Consume actions,my policy setting as below:

my environment is not a kerberized cluster and also I did not observe any records in Access of Audit,any suggestion?

thanks.

Re: Ranger policy malfunction in kafka

bensonshih — Mon, 01 Feb 2016 17:18:14 GMT

BTW, the following steps are how I enabled ranger for kafka and executed Publish/Consume actions:

1. In kafka Configs > Advanced ranger-kafka-aduit > enable "Audit to DB" and changed value of "xasecure.audit.destination.hdfs.dir" to "hdfs://140.92.XX.XX:8020/ranger/audit"

2. Configs > Advanced ranger-kafka-plugin-properties > enable "Enable Ranger for KAFKA"

3. save changes and restart KAFKA

4. go to Ranger admin UI and I saw the repository of kafka has been created automatically

Re: Ranger policy malfunction in kafka

bensonshih — Mon, 01 Feb 2016 17:18:46 GMT

Executing consume/publish steps:

Step1: connect to kafka-broker server

step2: changer user $ su kafka

step3: go to bin folder $ cd /usr/hdp/2.3.0.0-2557/kafka/bin

step4: create a topic $ ./kafka-topics.sh --create --zookeeper {hostname}:2181 --replication-factor 1 --partitions 1 --topic test

step5: execute publish message $ ./kafka-console-producer.sh --broker-list {hostname}:6667--topic test

This is a test message //it should be denied right?

step6: execute consume message $ ./kafka-console-consumer.sh --zookeeper {hostname}:2181 --topic test --from-beginning

//it also should be denied?

Re: Ranger policy malfunction in kafka

nsabharwal — Mon, 01 Feb 2016 19:21:02 GMT

@Benson Shih I really appreciate you sharing the details.

In the Ranger policy, Did you set the IP?

Can I authorize access to Kafka over a non-secure channel via Ranger?

Yes. you can control access by ip-address.