Multi-tenant NiFi writing to kerberized HDFS and Kafka

[#1]

Apologies for asking two questions at once, but they're very closely related. I have a NiFi cluster configured with kerberos and SSL that I would like to use with multiple tenants with strict policies in order to keep different groups from reading and/or altering each others' data. Each of the tenants need to write to HDFS and/or Kafka (which are in turn both kerberized).

HDFS

The PutHDFS processor allows me to specify a kerberos principal and keytab, but that implies the linux user running the NiFi process has to have read access to the keytab, which means any NiFi tenant could specify any keytab as long as they knew the path. Is there a way to keep those identities distinct and secure?

Kafka

The PutKafka processors only allow referencing the Kerberos Service Name from a jaas file that NiFi needs to know about at startup. That suggests to me that all tenants in NiFi would have to connect to Kafka as the same principal. Is there a way to do this that allows each tenant to have their own identity when talking to Kafka?

Thanks in advance for any tips!

[#2 · Accepted solution]

Hi @Oliver Meyn,

You are absolutely right on all points. Regarding HDFS, at the moment there is no option to keep the identities secure, but there are discussions going on to find a way to secure the keytabs.

Regarding Kafka, this is due to a limitation on Kafka's side (https://issues.apache.org/jira/browse/KAFKA-4259). This has been recently fixed and I'm sure the Kafka processors in NiFi will be updated in the future to allow a similar option as in other processors.

Hope this helps.

[#3]

Hi all, hi @Pierre Villard,

I have the same question as Olivier last year, do you know whether there is any improvement of the keytab accessiblity in NIFI?

Thanks in advance

Arne

[#4]

This seems to be the desired solution...

https://bryanbende.com/development/2018/04/09/apache-nifi-secure-keytab-access