Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Multi-tenant NiFi writing to kerberized HDFS and Kafka

Solved Go to solution
Highlighted

Multi-tenant NiFi writing to kerberized HDFS and Kafka

Explorer

Apologies for asking two questions at once, but they're very closely related. I have a NiFi cluster configured with kerberos and SSL that I would like to use with multiple tenants with strict policies in order to keep different groups from reading and/or altering each others' data. Each of the tenants need to write to HDFS and/or Kafka (which are in turn both kerberized).

HDFS

The PutHDFS processor allows me to specify a kerberos principal and keytab, but that implies the linux user running the NiFi process has to have read access to the keytab, which means any NiFi tenant could specify any keytab as long as they knew the path. Is there a way to keep those identities distinct and secure?

Kafka

The PutKafka processors only allow referencing the Kerberos Service Name from a jaas file that NiFi needs to know about at startup. That suggests to me that all tenants in NiFi would have to connect to Kafka as the same principal. Is there a way to do this that allows each tenant to have their own identity when talking to Kafka?

Thanks in advance for any tips!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Multi-tenant NiFi writing to kerberized HDFS and Kafka

Hi @Oliver Meyn,

You are absolutely right on all points. Regarding HDFS, at the moment there is no option to keep the identities secure, but there are discussions going on to find a way to secure the keytabs.

Regarding Kafka, this is due to a limitation on Kafka's side (https://issues.apache.org/jira/browse/KAFKA-4259). This has been recently fixed and I'm sure the Kafka processors in NiFi will be updated in the future to allow a similar option as in other processors.

Hope this helps.

View solution in original post

3 REPLIES 3
Highlighted

Re: Multi-tenant NiFi writing to kerberized HDFS and Kafka

Hi @Oliver Meyn,

You are absolutely right on all points. Regarding HDFS, at the moment there is no option to keep the identities secure, but there are discussions going on to find a way to secure the keytabs.

Regarding Kafka, this is due to a limitation on Kafka's side (https://issues.apache.org/jira/browse/KAFKA-4259). This has been recently fixed and I'm sure the Kafka processors in NiFi will be updated in the future to allow a similar option as in other processors.

Hope this helps.

View solution in original post

Highlighted

Re: Multi-tenant NiFi writing to kerberized HDFS and Kafka

New Contributor

Hi all, hi @Pierre Villard,

I have the same question as Olivier last year, do you know whether there is any improvement of the keytab accessiblity in NIFI?

Thanks in advance

Arne

Highlighted

Re: Multi-tenant NiFi writing to kerberized HDFS and Kafka

New Contributor
Don't have an account?
Coming from Hortonworks? Activate your account here