use of nifi with Kafka and Ranger

[#1]

Hey, folks,

I have a Kerberized server, and for the police I'm using Ranger, via console I can publish and consume the messages of Kafka, But now I must do through Nifi and that I'm very lost someone could help me.

Thank you

[#2]

@Peruvian81 

You can start testing a flow like below:

tailFile --> PublishKafka_1_0(2_0 depending on your Kafka version) 

In publishKafka you can use a configuration example like below:

• Ensure that the principal has Ranger authorization to publish data to the topic.
• In Kafka brokers, provide the brokers FQDM, do not use localhost or IPs

[#3]

Hello ManuelCalvo

I don't know if it's asking you too much, as I would have to do a relationship of publisher and consumer. I'm a little new to the subject of Big data. Thank you very much for the info.

Greetings

[#4 · Accepted solution]

@Peruvian81 

You can try below flow which is just for testing purposes:

Basically I have a tailFile processor passing data through splitText then these messages are sent to PublishKafka_1_0(use this processor for this test), finally I created a consumer to consume data from the same topic configured in PublishKafka_1_0 storing the data in the file system with putFile.

In putFile I have configured Maximum File Count to 10, to avoid excessive space usage in the file system.