Oh no, I forgot or lost my NiFi sensitive.props.key, what can I do?

[#1]

Objective

This article will share what options a user has when they have lost or forgot the password set for the nifi.sensitive.props.key in the nifi.properties file.

What is the nifi.sensitive.props.key?
----------------------------------------------

The nifi.sensitive.props.key is used to encode any NiFi component added to the NiFi's canvas that has a sensitive property configured with a value (passwords). These "enc{...}" strings are set in the flow.xml.gz that is generated by NIFi, which includes everything dataflow-related set through the NiFi UI. How the nifi.sensitive.props.key gets set has varied over the history of NiFi. In some releases, NiFi would automatically generate a random password when the user did not set the property when NiFi was first launched. The current release requires that a user set this property or NiFi will not start. The nifi.sensitive.props.key is coupled closely with the flow.xml.gz. The flow.xml.gz file can be migrated to other NiFi installations, but in order for that other instance to load that flow.xml.gz, one of the following must be true:

• The flow.xml.gz includes no encrypted passwords
• The NiFi loading the flow.xml.gz must be configured with the same nifi.sensitive.props.key, as the NiFi where the flow.xml.gz was obtained

What options do I have if can't remember or lost my nifi.sensitive.props.key set password?

----------------------------------------------

NiFi does not store the nifi.sensitive.props.key anywhere unencrypted; however there are two paths available to move beyond this issue so that the flow.xml.gz can be loaded.

• OPTION 1: Since the nifi.sensitive.props.key is used to unencrypt the sensitive property values within the flow.xml.gz, a user could edit the flow.xml.gz in order to remove all the configured sensitive property values. Then any nifi.sensitive.props.key can be set and NiFi will be able to start. 
  • Advantages:
    • All you need is the flow.xml.gz file
  • Disadvantages:
    • Clearing all the sensitive properties values will require those values to be re-entered via the NiFi UI after the flow.xml.gz is successfully loaded.
  • Process:
    • Use a command-line editor like VIM to locate all occurrences of "enc{.*}"
      Example: 

      vim flow.xml.gz
      :%s/enc{.*}//g

  • Use the flow.xml.gz in your new NiFi setup with whatever new nifi.senstive.props.key value you want. As you re-enter sensitive property values in your existing dataflows, they will be written in an encrypted format to the flow.xml.gz using the new nifi.senstive.props.key password you configured.

• OPTION 2: NiFi provides a toolkit that allows you to change the nifi.sensitive.props.key and re-encrypt the sensitive properties in the flow.xml.gz at the same time.
  • Advantages:
    • No need to re-enter all your passwords in the flow.xml.gz 
  • Disadvantages/challenges:
    • Need access to the following configuration files from original NiFi that goes with the flow.xml.gz
      • nifi.properties
      • bootstrap.conf
      • flow.xml.gz
  • Process:
    • Use the NiFi-toolkit "encrypt-config.sh" script. Cloudera includes the nifi-toolkit on each host where NiFi is deployed; the toolkit can also be downloaded from the Apache NiFi site.
    • Collect copies of the nifi.properties, bootstrap.conf, and flow.xml.gz files from original source working NiFi.
    • Execute the following toolkit:

      ./encrypt-config.sh -v -n /<path to copy of>/nifi.properties -o /tmp/nifi.properties.new -b /<path to copy of>/bootstrap.conf -f /<path to copy of>/flow.xml.gz -g /tmp/flow.xml.gz -s <newpassword> -x​

    • After the above command is executed, you will have two new files written out to /tmp:
      
      • A new nifi.properties file with the new password (encoded or plaintext depending on bootstrap.conf configuration) set.
      • new flow.xml.gz with all "enc{...}" re-encoded to using the new nifi.sensitive.props.key provided in the above command.
    • Use the nifi.senstive.props.* property values (4 of them) from the new nifi.properties file and the flow.xml.gz in your target NiFi.

Thank you,

Matt