Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi Won't Start after bootstrap.conf modified

Highlighted

Nifi Won't Start after bootstrap.conf modified

New Contributor

I have a reproducible bug where nifi service won't start. Beginning from a clean HDF install, I change the JVM memory settings in Ambari UI - nifi-bootstrap-env. (I have 32G of memory on my host).


# JVM memory settings

java.arg.2=-Xms{ {nifi_initial_mem}}

java.arg.3=-Xmx{ {nifi_max_mem}}


to

# JVM memory settings

java.arg.2=-Xms12228m

java.arg.3=-Xmx12228m


Then the restart services is successful with no errors, but the service doesn't actually start (in Ambari UI summary it shows the component as stopped). Even if I revert the configs back, I get the same bug - Nifi won't start.


stderr: /var/lib/ambari-agent/data/errors-225.txt

None

stdout: /var/lib/ambari-agent/data/output-225.txt

2019-06-03 14:34:05,743 - Stack Feature Version Info: Cluster Stack=3.4, Command Stack=None, Command Version=3.4.1.1-4 -> 3.4.1.1-4
2019-06-03 14:34:05,863 - Stack Feature Version Info: Cluster Stack=3.4, Command Stack=None, Command Version=3.4.1.1-4 -> 3.4.1.1-4
2019-06-03 14:34:05,866 - Group['nifiregistry'] {}
2019-06-03 14:34:05,867 - Group['hadoop'] {}
2019-06-03 14:34:05,867 - Group['nifi'] {}
2019-06-03 14:34:05,867 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-06-03 14:34:05,868 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-06-03 14:34:05,869 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-06-03 14:34:05,869 - User['nifiregistry'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['nifiregistry'], 'uid': None}
2019-06-03 14:34:05,870 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['nifi'], 'uid': None}
2019-06-03 14:34:05,870 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2019-06-03 14:34:05,871 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2019-06-03 14:34:05,875 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2019-06-03 14:34:05,888 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2019-06-03 14:34:05,891 - Skipping Execute[('setenforce', '0')] due to not_if
2019-06-03 14:34:05,891 - Testing the JVM's JCE policy to see it if supports an unlimited key length.
2019-06-03 14:34:06,068 - The unlimited key JCE policy is required, and appears to have been installed.
2019-06-03 14:34:06,329 - Stack Feature Version Info: Cluster Stack=3.4, Command Stack=None, Command Version=3.4.1.1-4 -> 3.4.1.1-4
2019-06-03 14:34:06,392 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2019-06-03 14:34:06,393 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_112;/usr/hdf/current/nifi/bin/nifi.sh stop >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'}
2019-06-03 14:34:09,855 - Pid file /var/run/nifi/nifi.pid is empty or does not exist
2019-06-03 14:34:09,856 - File['/var/lib/ambari-agent/tmp/run_ca.sh'] {'owner': 'nifi', 'content': StaticFile('run_ca.sh'), 'group': 'nifi', 'mode': 0755}
2019-06-03 14:34:09,856 - Changing owner of package files
2019-06-03 14:34:09,856 - Directory['/usr/hdf/current/nifi-toolkit'] {'group': 'nifi', 'cd_access': 'a', 'recursion_follow_links': True, 'create_parents': False, 'recursive_ownership': True, 'owner': 'nifi', 'mode': 0755}
2019-06-03 14:34:09,862 - Directory['/var/run/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,863 - Directory['/var/lib/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,995 - Directory['/var/lib/nifi/database_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,995 - Directory['/var/lib/nifi/flowfile_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,996 - Directory['/var/lib/nifi/provenance_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,997 - Directory['/usr/hdf/current/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,998 - Directory['/var/lib/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,998 - Directory['/var/lib/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:09,999 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:10,002 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:10,029 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True, 'cd_access': 'a'}
2019-06-03 14:34:10,057 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2019-06-03 14:34:10,060 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2019-06-03 14:34:10,061 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2019-06-03 14:34:10,065 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2019-06-03 14:34:10,065 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600, 'encoding': 'UTF-8'}
2019-06-03 14:34:10,163 - Writing File['/usr/hdf/current/nifi/conf/nifi.properties'] because contents don't match
2019-06-03 14:34:10,168 - File['/usr/hdf/current/nifi/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2019-06-03 14:34:10,169 - Writing File['/usr/hdf/current/nifi/conf/bootstrap.conf'] because contents don't match
2019-06-03 14:34:10,174 - File['/usr/hdf/current/nifi/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2019-06-03 14:34:10,177 - File['/usr/hdf/current/nifi/conf/state-management.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2019-06-03 14:34:10,186 - File['/usr/hdf/current/nifi/conf/authorizers.xml'] {'owner': 'nifi', 'content': ..., 'group': 'nifi', 'mode': 0600}
2019-06-03 14:34:10,191 - File['/usr/hdf/current/nifi/conf/login-identity-providers.xml'] {'owner': 'nifi', 'content': '<loginIdentityProviders>\n            \n            \n            \n            \n            \n            \n\n            </loginIdentityProviders>', 'group': 'nifi', 'mode': 0600}
2019-06-03 14:34:10,194 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2019-06-03 14:34:10,197 - File['/usr/hdf/current/nifi/conf/bootstrap-notification-services.xml'] {'owner': 'nifi', 'content': '<services>\n            \n            \n\n            \n\n            \n            </services>', 'group': 'nifi', 'mode': 0400}
2019-06-03 14:34:10,197 - Using repository toolkit script: /usr/hdf/current/nifi-toolkit/bin/encrypt-config.sh
2019-06-03 14:34:10,197 - File['/usr/hdf/current/nifi-toolkit/bin/encrypt-config.sh'] {'mode': 0755}
2019-06-03 14:34:10,197 - Encrypting NiFi sensitive configuration properties
2019-06-03 14:34:10,201 - Execute[(u'/usr/hdf/current/nifi-toolkit/bin/encrypt-config.sh', '-v', '-b', u'/usr/hdf/current/nifi/conf/bootstrap.conf', '-n', u'/usr/hdf/current/nifi/conf/nifi.properties', '-f', u'/var/lib/nifi/conf/flow.xml.gz', '-s', [PROTECTED], '-a', u'/usr/hdf/current/nifi/conf/authorizers.xml', '-m', '-e', [PROTECTED], '-p', [PROTECTED])] {'environment': {'JAVA_OPTS': u'-Xms128m -Xmx256m', 'JAVA_HOME': u'/usr/jdk64/jdk1.8.0_112'}, 'logoutput': False, 'user': 'nifi'}
2019-06-03 14:34:12,719 - ModifyPropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'properties': {'#nifi.security.ambari.hash.kspwd': '32b68561e3f9aae8cf3871dc16db6e717759449332926e0cfed0005b5bea282a', '#nifi.security.ambari.hash.kpwd': '32b68561e3f9aae8cf3871dc16db6e717759449332926e0cfed0005b5bea282a', '#nifi.security.ambari.hash.tspwd': '32b68561e3f9aae8cf3871dc16db6e717759449332926e0cfed0005b5bea282a'}}
2019-06-03 14:34:12,719 - Modifying existing properties file: /usr/hdf/current/nifi/conf/nifi.properties
2019-06-03 14:34:12,722 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': ..., 'group': None, 'mode': None, 'encoding': 'utf-8'}
2019-06-03 14:34:12,722 - Writing File['/usr/hdf/current/nifi/conf/nifi.properties'] because contents don't match
2019-06-03 14:34:12,725 - Directory['/usr/lib/ambari-logsearch-logfeeder/conf'] {'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2019-06-03 14:34:12,726 - Generate Log Feeder config file: /usr/lib/ambari-logsearch-logfeeder/conf/input.config-nifi.json
2019-06-03 14:34:12,726 - File['/usr/lib/ambari-logsearch-logfeeder/conf/input.config-nifi.json'] {'content': Template('input.config-nifi.json.j2'), 'mode': 0644}
2019-06-03 14:34:12,726 - Ranger admin not installed
2019-06-03 14:34:12,726 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_112;/usr/hdf/current/nifi/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'}
2019-06-03 14:34:15,744 - Execute['sleep 5'] {}
2019-06-03 14:34:20,748 - Trying to register NIFI Registry. This can take up to several minutes. Please, wait...
2019-06-03 14:34:20,748 - Trying to retrieve NIFI clients...
2019-06-03 14:34:20,748 - Using repository toolkit script: /usr/hdf/current/nifi-toolkit/bin/cli.sh
2019-06-03 14:34:20,748 - Using repository toolkit script: /usr/hdf/current/nifi-toolkit/bin/encrypt-config.sh
2019-06-03 14:36:24,932 - Unable to create or update registry client. Please review NiFi to setup the registry manually if necessary.
2019-06-03 14:36:24,932 - Pid files for current script are not defined

Command completed successfully!


/var/log/nifi/nifi-setup.log

2019-06-03 14:34:06,849 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi is not currently running


Java home: /usr/jdk64/jdk1.8.0_112
NiFi home: /usr/hdf/current/nifi

Bootstrap Config File: /usr/hdf/current/nifi/conf/bootstrap.conf