Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

use of nifi with Kafka and Ranger

Solved Go to solution
Highlighted

use of nifi with Kafka and Ranger

Explorer

Hey, folks,

I have a Kerberized server, and for the police I'm using Ranger, via console I can publish and consume the messages of Kafka, But now I must do through Nifi and that I'm very lost someone could help me.

Thank you

1 ACCEPTED SOLUTION

Accepted Solutions

Re: use of nifi with Kafka and Ranger

Contributor

@Peruvian81 

 

You can try below flow which is just for testing purposes:

 

Screen Shot 2019-10-08 at 1.56.03 PM.png

 

Basically I have a tailFile processor passing data through splitText then these messages are sent to PublishKafka_1_0(use this processor for this test), finally I created a consumer to consume data from the same topic configured in PublishKafka_1_0 storing the data in the file system with putFile.

 

In putFile I have configured Maximum File Count to 10, to avoid excessive space usage in the file system.

3 REPLIES 3

Re: use of nifi with Kafka and Ranger

Contributor

@Peruvian81 

 

You can start testing a flow like below:

 

tailFile --> PublishKafka_1_0(2_0 depending on your Kafka version) 

 

In publishKafka you can use a configuration example like below:

 

Screen Shot 2019-10-07 at 3.58.07 PM.png

 

  • Ensure that the principal has Ranger authorization to publish data to the topic.
  • In Kafka brokers, provide the brokers FQDM, do not use localhost or IPs

 

Re: use of nifi with Kafka and Ranger

Explorer

 

Hello ManuelCalvo

 

I don't know if it's asking you too much, as I would have to do a relationship of publisher and consumer. I'm a little new to the subject of Big data. Thank you very much for the info.

 

Greetings

Re: use of nifi with Kafka and Ranger

Contributor

@Peruvian81 

 

You can try below flow which is just for testing purposes:

 

Screen Shot 2019-10-08 at 1.56.03 PM.png

 

Basically I have a tailFile processor passing data through splitText then these messages are sent to PublishKafka_1_0(use this processor for this test), finally I created a consumer to consume data from the same topic configured in PublishKafka_1_0 storing the data in the file system with putFile.

 

In putFile I have configured Maximum File Count to 10, to avoid excessive space usage in the file system.

Don't have an account?
Coming from Hortonworks? Activate your account here