HDF3 install via blueprint, NiFi fails to start due to "Password expected" at decrypting flow.xml.gz

[#1]

Hi,

I am setting up HDF3.0 via Blueprint where installation of services works nice, but starting NiFi fails because it expects a password being provided for decrypting flow.xml.gz under /nifi directory (calling the encrypt-config tool).

Two questions here:

1.) which properties needs to be provided in the blueprint so that NiFi starts successfully without being asked for a password....which obviously cannot be provided

2.) from where does NiFi at startup time populate the subdirectories under /nifi ? Before re-deploying the blueprint I deleted whole /nifi directory just to ensure the main error is not caused by some old/previous files, but at starting up Nifi this folder gets recreated incl. subdirectories.

Any other hint to get the services startup also being successfull at applying the blueprint highly appreciated 😉

[#2 · Accepted solution]

Hi @Gerd Koenig

NiFi needs two configuration passwords, nifi.security.encrypt.configuration.password and nifi.sensitive.props.key. You can pass those using the nifi-ambari-config in the blueprint. The passwords also need to be at least 12 characters in length.

Here's a simple blueprint that normally works for me:

{
  "configurations" : [
    {
      "nifi-ambari-config" : {
        "nifi.node.ssl.port": "9091",
        "nifi.node.port": "9090",
        "nifi.security.encrypt.configuration.password": "AsdQwe123456",
        "nifi.sensitive.props.key": "AsdQwe123456"
      }
    },
    {
      "nifi-env" : {
        "nifi_group" : "nifi",
        "nifi_user" : "nifi"
      }
    }
  ],
  "host_groups" : [
    {
      "name" : "mytestcluster-singlenode",
      "configurations" : [ ],
      "components" : [
        { "name" : "ZOOKEEPER_CLIENT" },
        { "name" : "INFRA_SOLR_CLIENT" },
        { "name" : "ZOOKEEPER_SERVER" },
        { "name" : "NIFI_MASTER" },
        { "name" : "AMBARI_SERVER" },
        { "name" : "INFRA_SOLR" },
        { "name" : "METRICS_COLLECTOR" },
        { "name" : "METRICS_GRAFANA" },
        { "name" : "METRICS_MONITOR" }
        
      ]
    }
  ],
  "Blueprints" : {
    "stack_name" : "HDF",
    "stack_version" : "3.0"
  }
}

Regarding the folders, yes, NiFi will recreate them based on the configuration variables. These can also be set in the blueprint:

    {
      "nifi-ambari-config" : {
          "nifi.internal.dir" : "/var/lib/nifi",
          "nifi.content.repository.dir.default" : "/var/lib/nifi/content_repository",
          "nifi.state.dir" : "{nifi_internal_dir}/state/local",
          "nifi.flow.config.dir" : "{nifi_internal_dir}/conf",
          "nifi.config.dir" : "{nifi_install_dir}/conf",
          "nifi.flowfile.repository.dir" : "/var/lib/nifi/flowfile_repository",
          "nifi.provenance.repository.dir.default" : "/var/lib/nifi/provenance_repository",
          "nifi.database.dir" : "/var/lib/nifi/database_repository"
      }
    },
    

[#3]

Hello @Alexandru Anghel ,

many thanks....works brilliant !