Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NIFI Start up Issue

avatar

I am getting below error while starting NIFI through Ambari.

 

Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 278, in <module>
Master().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 351, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 142, in start
self.configure(env, is_starting = True)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 110, in configure
self.write_configurations(params, is_starting)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 227, in write_configurations
params.stack_support_encrypt_authorizers, params.stack_version_buildnum)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi_toolkit_util.py", line 393, in encrypt_sensitive_properties
Execute(encrypt_config_command, user=nifi_user, logoutput=False, environment=environment)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/var/lib/ambari-agent/tmp/nifi-toolkit-1.7.0.3.2.0.0-520/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -f /var/lib/nifi/conf/flow.xml.gz -s '[PROTECTED]' -a /usr/hdf/current/nifi/conf/authorizers.xml -p '[PROTECTED]'' returned 255. 2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of nifi.properties
2020/01/22 02:27:24 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source nifi.properties and destination nifi.properties are identical [/usr/hdf/current/nifi/conf/nifi.properties] so the original will be overwritten
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of authorizers.xml
2020/01/22 02:27:24 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source authorizers.xml and destination authorizers.xml are identical [/usr/hdf/current/nifi/conf/authorizers.xml] so the original will be overwritten
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of flow.xml.gz
2020/01/22 02:27:24 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source flow.xml.gz and destination flow.xml.gz are identical [/var/lib/nifi/conf/flow.xml.gz] so the original will be overwritten
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: bootstrap.conf: /usr/hdf/current/nifi/conf/bootstrap.conf
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src) nifi.properties: /usr/hdf/current/nifi/conf/nifi.properties
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties: /usr/hdf/current/nifi/conf/nifi.properties
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src) login-identity-providers.xml: null
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) login-identity-providers.xml: null
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src) authorizers.xml: /usr/hdf/current/nifi/conf/authorizers.xml
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) authorizers.xml: /usr/hdf/current/nifi/conf/authorizers.xml
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src) flow.xml.gz: /var/lib/nifi/conf/flow.xml.gz
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz: /var/lib/nifi/conf/flow.xml.gz
2020/01/22 02:27:24 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 154 properties from /usr/hdf/current/nifi/conf/nifi.properties
2020/01/22 02:27:25 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 154 properties from /usr/hdf/current/nifi/conf/nifi.properties
2020/01/22 02:27:26 INFO [main] org.apache.nifi.properties.ProtectedNiFiProperties: There are 1 protected properties of 5 sensitive properties (100%)
2020/01/22 02:27:26 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Loaded NiFiProperties instance with 153 properties
2020/01/22 02:27:26 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Loaded authorizers content (47 lines)
2020/01/22 02:27:26 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: No encrypted password property elements found in authorizers.xml
2020/01/22 02:27:26 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: No unencrypted password property elements found in authorizers.xml
2020/01/22 02:27:26 ERROR [main] org.apache.nifi.toolkit.encryptconfig.EncryptConfigMain:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:596)
at java.lang.StringBuilder.append(StringBuilder.java:190)
at org.apache.commons.io.output.StringBuilderWriter.write(StringBuilderWriter.java:142)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2538)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2516)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2493)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2441)
at org.apache.commons.io.IOUtils.toString(IOUtils.java:1084)
at org.apache.commons.io.IOUtils$toString.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
at org.apache.nifi.properties.ConfigEncryptionTool$_loadFlowXml_closure3$_closure29.doCall(ConfigEncryptionTool.groovy:666)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at groovy.lang.Closure.call(Closure.java:414)
at groovy.lang.Closure.call(Closure.java:430)
at org.codehaus.groovy.runtime.IOGroovyMethods.withCloseable(IOGroovyMethods.java:1622)
at org.codehaus.groovy.runtime.NioGroovyMethods.withCloseable(NioGroovyMethods.java:1759)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.metaclass.ReflectionMetaMethod.invoke(ReflectionMetaMethod.java:54)
Java heap space

usage: org.apache.nifi.toolkit.encryptconfig.EncryptConfigMain [-h] [options]

This tool enables easy encryption and decryption of configuration files for NiFi and its sub-projects. Unprotected files can be input to this tool to be
protected by a key in a manner that is understood by NiFi. Protected files, along with a key, can be input to this tool to be unprotected, for troubleshooting
or automation purposes.

-h,--help Show usage information (this message)
--nifiRegistry Specifies to target NiFi Registry. When this flag is not included, NiFi is the target.

When targeting NiFi:
-h,--help Show usage information (this message)
-v,--verbose Sets verbose mode (default false)
-n,--niFiProperties <file> The nifi.properties file containing unprotected config values (will be overwritten unless -o is specified)
-o,--outputNiFiProperties <file> The destination nifi.properties file containing protected config values (will not modify input nifi.properties)
-l,--loginIdentityProviders <file> The login-identity-providers.xml file containing unprotected config values (will be overwritten unless -i is
specified)
-i,--outputLoginIdentityProviders <file> The destination login-identity-providers.xml file containing protected config values (will not modify input
login-identity-providers.xml)
-a,--authorizers <file> The authorizers.xml file containing unprotected config values (will be overwritten unless -u is specified)
-u,--outputAuthorizers <file> The destination authorizers.xml file containing protected config values (will not modify input authorizers.xml)
-f,--flowXml <file> The flow.xml.gz file currently protected with old password (will be overwritten unless -g is specified)
-g,--outputFlowXml <file> The destination flow.xml.gz file containing protected config values (will not modify input flow.xml.gz)
-b,--bootstrapConf <file> The bootstrap.conf file to persist master key
-k,--key <keyhex> The raw hexadecimal key to use to encrypt the sensitive properties
-e,--oldKey <keyhex> The old raw hexadecimal key to use during key migration
-p,--password <password> The password from which to derive the key to use to encrypt the sensitive properties
-w,--oldPassword <password> The old password from which to derive the key during migration
-r,--useRawKey If provided, the secure console will prompt for the raw key value in hexadecimal form
-m,--migrate If provided, the nifi.properties and/or login-identity-providers.xml sensitive properties will be re-encrypted with
a new key
-x,--encryptFlowXmlOnly If provided, the properties in flow.xml.gz will be re-encrypted with a new key but the nifi.properties and/or
login-identity-providers.xml files will not be modified
-s,--propsKey <password|keyhex> The password or key to use to encrypt the sensitive processor properties in flow.xml.gz
-A,--newFlowAlgorithm <algorithm> The algorithm to use to encrypt the sensitive processor properties in flow.xml.gz
-P,--newFlowProvider <algorithm> The security provider to use to encrypt the sensitive processor properties in flow.xml.gz
-c,--translateCli Translates the nifi.properties file to a format suitable for the NiFi CLI tool

When targeting NiFi Registry using the --nifiRegistry flag:
-h,--help Show usage information (this message)
-v,--verbose Sets verbose mode (default false)
-p,--password <password> Protect the files using a password-derived key. If an argument is not provided to this flag, interactive mode will
be triggered to prompt the user to enter the password.
-k,--key <keyhex> Protect the files using a raw hexadecimal key. If an argument is not provided to this flag, interactive mode will be
triggered to prompt the user to enter the key.
--oldPassword <password> If the input files are already protected using a password-derived key, this specifies the old password so that the
files can be unprotected before re-protecting.
--oldKey <keyhex> If the input files are already protected using a key, this specifies the raw hexadecimal key so that the files can
be unprotected before re-protecting.
-b,--bootstrapConf <file> The bootstrap.conf file containing no master key or an existing master key. If a new password or key is specified
(using -p or -k) and no output bootstrap.conf file is specified, then this file will be overwritten to persist the
new master key.
-B,--outputBootstrapConf <file> The destination bootstrap.conf file to persist master key. If specified, the input bootstrap.conf will not be
modified.
-r,--nifiRegistryProperties <file> The nifi-registry.properties file containing unprotected config values, overwritten if no output file specified.
-R,--outputNifiRegistryProperties <file> The destination nifi-registry.properties file containing protected config values.
-a,--authorizersXml <file> The authorizers.xml file containing unprotected config values, overwritten if no output file specified.
-A,--outputAuthorizersXml <file> The destination authorizers.xml file containing protected config values.
-i,--identityProvidersXml <file> The identity-providers.xml file containing unprotected config values, overwritten if no output file specified.
-I,--outputIdentityProvidersXml <file> The destination identity-providers.xml file containing protected config values.
--decrypt Can be used with -r to decrypt a previously encrypted NiFi Registry Properties file. Decrypted content is printed to
STDOUT.
stdout: /var/lib/ambari-agent/data/output-10212.txt

2020-01-22 02:27:21,104 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=3.0.1.0-187 -> 3.0.1.0-187
2020-01-22 02:27:21,125 - Using hadoop conf dir: /usr/hdp/3.0.1.0-187/hadoop/conf
2020-01-22 02:27:21,367 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=3.0.1.0-187 -> 3.0.1.0-187
2020-01-22 02:27:21,373 - Using hadoop conf dir: /usr/hdp/3.0.1.0-187/hadoop/conf
2020-01-22 02:27:21,375 - Group['livy'] {}
2020-01-22 02:27:21,377 - Group['spark'] {}
2020-01-22 02:27:21,378 - Group['ranger'] {}
2020-01-22 02:27:21,378 - Group['nifiregistry'] {}
2020-01-22 02:27:21,378 - Group['hdfs'] {}
2020-01-22 02:27:21,378 - Group['zeppelin'] {}
2020-01-22 02:27:21,378 - Group['hadoop'] {}
2020-01-22 02:27:21,379 - Group['nifi'] {}
2020-01-22 02:27:21,379 - Group['users'] {}
2020-01-22 02:27:21,380 - User['yarn-ats'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,382 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,384 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,386 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,387 - User['superset'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,389 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,391 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['ranger', 'hadoop'], 'uid': None}
2020-01-22 02:27:21,392 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2020-01-22 02:27:21,394 - User['nifiregistry'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['nifiregistry'], 'uid': None}
2020-01-22 02:27:21,396 - User['zeppelin'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['zeppelin', 'hadoop'], 'uid': None}
2020-01-22 02:27:21,397 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['nifi'], 'uid': None}
2020-01-22 02:27:21,399 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['livy', 'hadoop'], 'uid': None}
2020-01-22 02:27:21,400 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['spark', 'hadoop'], 'uid': None}
2020-01-22 02:27:21,402 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2020-01-22 02:27:21,403 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop'], 'uid': None}
2020-01-22 02:27:21,405 - User['sqoop'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,406 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,408 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2020-01-22 02:27:21,409 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2020-01-22 02:27:21,415 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2020-01-22 02:27:21,424 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2020-01-22 02:27:21,425 - Group['hdfs'] {}
2020-01-22 02:27:21,426 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop', u'hdfs']}
2020-01-22 02:27:21,427 - FS Type: HDFS
2020-01-22 02:27:21,427 - Directory['/etc/hadoop'] {'mode': 0755}
2020-01-22 02:27:21,446 - File['/usr/hdp/3.0.1.0-187/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2020-01-22 02:27:21,449 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2020-01-22 02:27:21,471 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2020-01-22 02:27:21,486 - Skipping Execute[('setenforce', '0')] due to not_if
2020-01-22 02:27:21,487 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'}
2020-01-22 02:27:21,490 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'}
2020-01-22 02:27:21,491 - Directory['/var/run/hadoop/hdfs'] {'owner': 'hdfs', 'cd_access': 'a'}
2020-01-22 02:27:21,491 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'}
2020-01-22 02:27:21,497 - File['/usr/hdp/3.0.1.0-187/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2020-01-22 02:27:21,503 - File['/usr/hdp/3.0.1.0-187/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2020-01-22 02:27:21,510 - File['/usr/hdp/3.0.1.0-187/hadoop/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2020-01-22 02:27:21,524 - File['/usr/hdp/3.0.1.0-187/hadoop/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2020-01-22 02:27:21,525 - File['/usr/hdp/3.0.1.0-187/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2020-01-22 02:27:21,530 - File['/usr/hdp/3.0.1.0-187/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2020-01-22 02:27:21,536 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop', 'mode': 0644}
2020-01-22 02:27:21,545 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2020-01-22 02:27:21,553 - Testing the JVM's JCE policy to see it if supports an unlimited key length.
2020-01-22 02:27:21,762 - The unlimited key JCE policy is required, and appears to have been installed.
2020-01-22 02:27:21,772 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure.
2020-01-22 02:27:22,183 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=3.0.1.0-187 -> 3.0.1.0-187
2020-01-22 02:27:22,262 - File['/var/lib/ambari-agent/tmp/run_ca.sh'] {'owner': 'nifi', 'content': StaticFile('run_ca.sh'), 'group': 'nifi', 'mode': 0755}
2020-01-22 02:27:22,270 - Directory['/var/run/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,271 - Directory['/var/lib/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,551 - Directory['/var/lib/nifi/database_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,552 - Directory['/var/lib/nifi/flowfile_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,552 - Directory['/var/lib/nifi/provenance_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,555 - Directory['/usr/hdf/current/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,556 - Directory['/var/lib/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,557 - Directory['/var/lib/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,558 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,562 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,637 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True, 'cd_access': 'a'}
2020-01-22 02:27:22,715 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2020-01-22 02:27:22,722 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2020-01-22 02:27:22,733 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2020-01-22 02:27:22,739 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2020-01-22 02:27:22,740 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600, 'encoding': 'UTF-8'}
2020-01-22 02:27:22,869 - Writing File['/usr/hdf/current/nifi/conf/nifi.properties'] because contents don't match
2020-01-22 02:27:22,874 - File['/usr/hdf/current/nifi/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2020-01-22 02:27:22,879 - File['/usr/hdf/current/nifi/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2020-01-22 02:27:22,885 - File['/usr/hdf/current/nifi/conf/state-management.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2020-01-22 02:27:22,896 - File['/usr/hdf/current/nifi/conf/authorizers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2020-01-22 02:27:22,904 - File['/usr/hdf/current/nifi/conf/login-identity-providers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2020-01-22 02:27:22,908 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2020-01-22 02:27:22,912 - File['/usr/hdf/current/nifi/conf/bootstrap-notification-services.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2020-01-22 02:27:22,914 - Encrypting NiFi sensitive configuration properties
2020-01-22 02:27:22,914 - File['/var/lib/ambari-agent/tmp/nifi-toolkit-1.7.0.3.2.0.0-520/bin/encrypt-config.sh'] {'mode': 0755}
2020-01-22 02:27:22,946 - Execute[('/var/lib/ambari-agent/tmp/nifi-toolkit-1.7.0.3.2.0.0-520/bin/encrypt-config.sh', '-v', '-b', u'/usr/hdf/current/nifi/conf/bootstrap.conf', '-n', u'/usr/hdf/current/nifi/conf/nifi.properties', '-f', u'/var/lib/nifi/conf/flow.xml.gz', '-s', [PROTECTED], '-a', u'/usr/hdf/current/nifi/conf/authorizers.xml', '-p', [PROTECTED])] {'environment': {'JAVA_OPTS': u'-Xms128m -Xmx256m', 'JAVA_HOME': u'/usr/jdk64/jdk1.8.0_112'}, 'logoutput': False, 'user': 'nifi'}
2020-01-22 02:27:26,884 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure.

Command failed after 1 tries

3 REPLIES 3

avatar
Super Guru

@saivenkatg55 Looks like “out of memory error”. You will need to update the min and max memory settings.  They are current 128m and 256m  Try 512m and 1024m and step up increments until it will start.  

 

If this reply solves your issue please mark this response as the accepted solution.

avatar
Explorer

Look in the nifi-app.log and the nifi-bootstrap.log, they'll provide more information on why NiFi didn't start up. The default location of these files will be in the nifi-root-directory/logs.

avatar
Super Mentor

@saivenkatg55 

 

Your Out Of Memory (OOM) issue is occurring in the NiFi toolkit script and not in NiFi itself.  Ambari will execute the NiFi toolkit before the NiFi service itself is ever started.  If the NiFi toolkit fails, Ambari does not proceed any further.

I can see you are running HDF 3.2.0 which exposes the heap memory settings under the NiFi configs in Ambari:
Screen Shot 2020-01-22 at 3.36.24 PM.png

Locate the above line and increase your heap settings here.  Once the toolkit script finished running which must occur before NiFi is even started, that JVM is gone.  So the memory allocation set here are short lived and will not impact available memory for your NiFi.

Hope this helps,

Matt