NiFi 2.8.0 Support for Retrieving Passwords from HashiCorp Vault KV During Startup

[#1]

Hi Team,

I am exploring secret management options in Apache NiFi 2.8.0 and would like to understand whether NiFi supports retrieving passwords directly from HashiCorp Vault KV during startup.

My use case is to store the following properties in Vault KV rather than in nifi.properties:

- nifi.security.keystorePasswd
- nifi.security.keyPasswd
- nifi.security.truststorePasswd

The objective is to avoid storing these sensitive values in plain text on the NiFi host while still allowing NiFi to access them during the bootstrap/startup process.

I am specifically interested in startup configuration properties that are required before the application is fully initialized.

Has anyone successfully configured NiFi 2.8.0 to retrieve these passwords directly from HashiCorp Vault KV during startup? If so, are there any recommended approaches, examples, or best practices for this use case?

Any guidance would be appreciated.

Thank you.

[#2]

Apache NiFi 2.8.0 fully supports retrieving sensitive startup properties such as:

nifi.security.keystorePasswd

nifi.security.keyPasswd

nifi.security.truststorePasswd

directly from HashiCorp Vault KV during bootstrap/startup, so these values do not need to be stored in plain text in nifi.properties.

HASHICORP_VAULT_KV Sensitive Property Provider

This uses the bootstrap-level Vault integration, which resolves secrets very early in the startup process

Configure conf/bootstrap-hashicorp-vault.conf (Vault URI, KV path, authentication method — e.g., Token, AppRole, or Certificate).

Link it in bootstrap.conf

nifi.bootstrap.protection.hashicorp.vault.conf=./conf/bootstrap-hashicorp-vault.conf

Store the passwords in Vault KV (v1) using paths like nifi-kv/default/nifi.security.keystorePasswd

Use the NiFi Toolkit encrypt-config tool with the HASHICORP_VAULT_KV protection scheme to replace plaintext values in nifi.properties with Vault references

Start NiFi — the bootstrap process will fetch the secrets from Vault

https://docs.cloudera.com/cfm/4.11.0/nifi-components-cfm/nifi-docs/toolkit-guide.html

Search for HASHICORP_VAULT_KV — explicitly lists it as a supported scheme with bootstrap config details. @TechStream 

[#3]

Hi @RAGHUY ,

Thank you for the information.

I have already implemented the HASHICORP_VAULT_KV Sensitive Property Provider configuration as described:

• Configured bootstrap-hashicorp-vault.conf

• Added nifi.bootstrap.protection.hashicorp.vault.conf in bootstrap.conf

• Stored the SSL passwords in Vault KV

• Used the encrypt-config toolkit (1.27) with the HASHICORP_VAULT_KV protection scheme to replace the plaintext values in nifi.properties

However, NiFi is still failing to start. The bootstrap process appears to resolve the secret reference, but when the server initializes, I get the following error:

ERROR [main] org.apache.nifi.web.server.JettyServer Failed to start Server
org.apache.nifi.security.ssl.BuilderConfigurationException: Key Store loading failed
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
Caused by: java.security.UnrecoverableKeyException: Password verification failed

This suggests that the password being used to load the keystore is either not being resolved correctly from Vault or does not match the actual keystore password.

To verify the Vault secret itself, I manually retrieved the password from Vault and used it with the keystore, and it appears to be correct.

Have you encountered this issue before with the HASHICORP_VAULT_KV protection scheme? Is there a recommended way to verify what value NiFi is resolving from Vault during bootstrap, or any known issues related to keystore/truststore password resolution in NiFi 2.8.0?

Thank you.

[#4]

Glad that you implemented.

The reference in nifi.properties is something like nifi-kv/default/nifi.security.keystorePasswd.
Inside Vault, the secret must have the password under the correct key name (usually value by default for KV v1 in NiFi's client).
If you stored it under a different key (e.g., password, secret, or custom), it won't pull the right value.

Extra spaces, newlines, or character encoding differences when the value was stored in Vault vs. the original plaintext password

For many PKCS12 keystores, nifi.security.keyPasswd must match nifi.security.keystorePasswd. If only one is protected and they differ slightly, or if one resolves and the other doesn't, this error appears.

Need to check the above. @TechStream