Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can NiFi integrate with hashicorp vault to store sensitive information

Solved Go to solution
Highlighted

How can NiFi integrate with hashicorp vault to store sensitive information

Explorer

We want to store our sensitive information such as passwords, private keys in Vault and retrieve it from Vault on need basis, kindly share your inputs which is much needed.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How can NiFi integrate with hashicorp vault to store sensitive information

Master Guru

@Umakanth 

 

NiFi components (Processors, controller services, reporting tasks, etc.) that have password properties do not support retrieving these passwords from an external source or service.  NiFi not only obscures the passwords in the UI, but also encrypts all those passwords when they are written to disk in the flow.xml.gz.  Additionally, for passwords entered in the various NiFi configuration files, NiFi offers and encrypt config toolkit that can encrypt all these sensitive properties in these configuration files on disk.

 

I suggest maybe opening an Apache NiFi Jira with details around what you are trying to accomplish here for a possible future feature.
- For passwords utilized through NiFi dataflow components, my thought here would be maybe around a NiFi controller service for connecting to such external services.  This would also require that processors that would need to use this new NiFi Controller Service to retrieve passwords would all need to be modified as well with new configuration properties to interface with the new controller service.   This is by no means a simple change in NiFi, but getting the idea out there with some strong use case for it can get the ball rolling in the community.

 

Hope this helps,

Matt

View solution in original post

2 REPLIES 2

Re: How can NiFi integrate with hashicorp vault to store sensitive information

Master Guru

@Umakanth 

 

NiFi components (Processors, controller services, reporting tasks, etc.) that have password properties do not support retrieving these passwords from an external source or service.  NiFi not only obscures the passwords in the UI, but also encrypts all those passwords when they are written to disk in the flow.xml.gz.  Additionally, for passwords entered in the various NiFi configuration files, NiFi offers and encrypt config toolkit that can encrypt all these sensitive properties in these configuration files on disk.

 

I suggest maybe opening an Apache NiFi Jira with details around what you are trying to accomplish here for a possible future feature.
- For passwords utilized through NiFi dataflow components, my thought here would be maybe around a NiFi controller service for connecting to such external services.  This would also require that processors that would need to use this new NiFi Controller Service to retrieve passwords would all need to be modified as well with new configuration properties to interface with the new controller service.   This is by no means a simple change in NiFi, but getting the idea out there with some strong use case for it can get the ball rolling in the community.

 

Hope this helps,

Matt

View solution in original post

Highlighted

Re: How can NiFi integrate with hashicorp vault to store sensitive information

Explorer

Thank you for sharing your thoughts around this integration, it really really helps.

I will try to do a POC based out of your suggestion

Don't have an account?
Coming from Hortonworks? Activate your account here