NiFi not starting in HDF Sandbox 3.1, cannot load flow.xml.gz

[#1]

I'm new to the Hortonworks environment and I'm using the HDF 3.1 Sandbox in Virtualbox. Since last night my flow.xml.gz won't load after a reboot. I haven't changed any variables from my perspective.

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 231, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 978, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 152, in start
    self.configure(env, is_starting = True)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 120, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 122, in configure
    params.stack_support_encrypt_authorizers)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi_toolkit_util.py", line 448, in encrypt_sensitive_properties
    Execute(encrypt_config_command, user=nifi_user, logoutput=False, environment=environment)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.0.0-564/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -f /var/lib/nifi/conf/flow.xml.gz -s '[PROTECTED]' -a /usr/hdf/current/nifi/conf/authorizers.xml -p '[PROTECTED]'' returned 9. The place it is looking is: /var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.0.0-564/classpath:/var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.0.0-564/lib/*
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of authorizers.xml
2018/03/29 09:44:18 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source authorizers.xml and destination authorizers.xml are identical [/usr/hdf/current/nifi/conf/authorizers.xml] so the original will be overwritten
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of nifi.properties
2018/03/29 09:44:18 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source nifi.properties and destination nifi.properties are identical [/usr/hdf/current/nifi/conf/nifi.properties] so the original will be overwritten
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of flow.xml.gz
2018/03/29 09:44:18 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source flow.xml.gz and destination flow.xml.gz are identical [/var/lib/nifi/conf/flow.xml.gz] so the original will be overwritten
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool:        bootstrap.conf:               /usr/hdf/current/nifi/conf/bootstrap.conf
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  nifi.properties:              /usr/hdf/current/nifi/conf/nifi.properties
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties:              /usr/hdf/current/nifi/conf/nifi.properties
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  login-identity-providers.xml: null
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) login-identity-providers.xml: null
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  authorizers.xml:              /usr/hdf/current/nifi/conf/authorizers.xml
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) authorizers.xml:              /usr/hdf/current/nifi/conf/authorizers.xml
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  flow.xml.gz:                  /var/lib/nifi/conf/flow.xml.gz
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz:                  /var/lib/nifi/conf/flow.xml.gz
2018/03/29 09:44:18 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 145 properties from /usr/hdf/current/nifi/conf/nifi.properties
2018/03/29 09:44:19 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 145 properties from /usr/hdf/current/nifi/conf/nifi.properties
2018/03/29 09:44:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Loaded NiFiProperties instance with 145 properties
2018/03/29 09:44:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Loaded authorizers content (74 lines)
2018/03/29 09:44:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: No encrypted password property elements found in authorizers.xml
2018/03/29 09:44:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: No unencrypted password property elements found in authorizers.xml
Cannot load flow.xml.gz

[#2]

How much memory are you allocating to VirtualBox? It should be no less than 8GB, but more is always better. I found that 10-11 GB works best.

[#3]

Thank you for the response, I've normally allocated 8GB, but I've booted up the machine with 12GB now. Still the same error log. I wonder what the returned 9 means in this part of the log.

resource_management.core.exceptions.ExecutionFailed: Execution of '/var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.0.0-564/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -f /var/lib/nifi/conf/flow.xml.gz -s '[PROTECTED]' -a /usr/hdf/current/nifi/conf/authorizers.xml -p '[PROTECTED]'' returned 9.

[#4]

@gdeleon

Matt posted an answer in HCC, You have encountered the following:

https://issues.apache.org/jira/browse/NIFI-4363

You can get around this issue by editing the following script on every NiFi node in your cluster:

/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.2.0.3.0.1.0-43/bin/encrypt-config.sh 

Near the bottom of the script you will find this line:

"${JAVA}"-cp "${CLASSPATH}"-Xms128m-Xmx256m org.apache.nifi.properties.ConfigEncryptionTool"$@"

Edit it so that it now says:

"${JAVA}"-cp "${CLASSPATH}"-Xms512m-Xmx1024m org.apache.nifi.properties.ConfigEncryptionTool"$@"

Then go back to Ambari and start your NiFi service.

Hope that helps

[#5]

I've ran into the same issue again after reverting to an older snapshot, I've updated the scripts and the config in Ambari (since the edits in the scripts didn't do anything and JAVA_OPTS needed a memory increase) but it didn't work out. Any suggestions are appreciated.

Maybe the flow.xml is corrupt?

Execute[('/var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.0.0-564/bin/encrypt-config.sh', '-v', '-b', u'/usr/hdf/current/nifi/conf/bootstrap.conf', '-n', u'/usr/hdf/current/nifi/conf/nifi.properties', '-f', u'/var/lib/nifi/conf/flow.xml.gz', '-s', [PROTECTED], '-a', u'/usr/hdf/current/nifi/conf/authorizers.xml', '-p', [PROTECTED])] {'environment': {'JAVA_OPTS': u'-Xms512m -Xmx1024m', 'JAVA_HOME': u'/usr/java/default'}, 'logoutput': False, 'user': 'nifi'}


Command failed after 1 tries

[#6]

Heads up folks!

The above responses are valid for previous version of HDF. For Ambari 2.6.1 with NIFI

1.5.0.3.1.1.0-35 or better the location changed to inside the Ambari interface. Please find

Ambari GUI --> NIFI ---> Configs ----> Advanced nifi-toolkit-env ---> NIFI Toolkit Java Options