NiFi - How to store a password and pass it to flowfile content

Question

I've developed a flow that depends on importing an Oracle database dump via data pump (i.e., the impdp command). To run this command, you have to pass in the username and password as part of the parameters of the impdp command to connect to the Oracle database. In early development, I simply put the password in a non-sensitive parameter, and at the appropriate time, I'm using a ReplaceText processor to wipe out the current contents of the flowfile and write in the contents of the shell script used to run the impdp command. Of course, the text I'm writing in to the flowfile includes #{oracle_pw}. I then PutFile to write the script to disk and ExecuteStreamCommand to run the script. It works like a charm.

Now that development is over and we're making plans to move this flow into production, I need a way to make this password parameter sensitive and still be able to pass it in to the script. I've given this a lot of thought and done some research, but it seems like I might need to reach beyond the confines of NiFi to make this happen. One thought was to ensure the OS account NiFi is using already has all the Oracle stuff in its path, and then instead of creating and running a script, pass the impdp command directly via ExecuteStreamCommand, but again, non-sensitive properties (i.e., command.argument.1, command.argument.2, etc.) cannot contain sensitive parameters.

I would appreciate any help in brainstorming how to accomplish this. Here are the basic contents of the script:

#!/bin/bash

eval "$(grep 'ORACLE_BASE=' /home/oracle/.bash_profile)"

eval "$(grep 'ORACLE_HOME=' /home/oracle/.bash_profile)"

PATH=$PATH:$ORACLE_HOME/bin

export ORACLE_BASE ORACLE_HOME PATH

impdp oracleuser/#{oracle_pw}@//oracleserver/oracledatabase more parameters, etc.

noncitizen · Accepted Answer

Thanks for the suggestions. I ended up moving everything to stored procedures in the database, which are run under the existing context (service), so no need for a sensitive parameter.

joseomjr · Answer

You should be able to create a custom scripted processor and define an attribute that's sensitive so once you set it...it won't be visible. You might be able to handle your whole process inside the script so the password never gets written out. I've written several in Groovy. An alternative option is to leverage something like Vault (Vault by HashiCorp (vaultproject.io)).

PropertyDescriptor OAUTH_TOKEN = new PropertyDescriptor.Builder()

.name("oAuth Token")

.displayName("oAuth Token")

.description("oAuth Token")

.required(true)

.sensitive(true)

.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)

.build()

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.