Created 08-20-2018 04:27 PM
Hey all, I am having issues getting Nifi and Nifi Registry to start so that they may be updated to 1.7 with the release of HDF 3.2. I'm not seeing many other topics covering this issue. I have checked that the file exists that is being referenced here, there is a directory for both Nifi 1.5 and 1.7 toolkit. I have tried moving the 1.7 directory out of the tmp folder in case that was creating confusion but that didn't work either. Any ideas?
stderr: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/NIFI_REGISTRY/0.1.0/package/scripts/nifi_registry.py", line 280, in <module> Master().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/NIFI_REGISTRY/0.1.0/package/scripts/nifi_registry.py", line 128, in start self.configure(env, is_starting = True) File "/var/lib/ambari-agent/cache/common-services/NIFI_REGISTRY/0.1.0/package/scripts/nifi_registry.py", line 98, in configure self.write_configurations(params, is_starting) File "/var/lib/ambari-agent/cache/common-services/NIFI_REGISTRY/0.1.0/package/scripts/nifi_registry.py", line 200, in write_configurations is_starting, params.toolkit_tmp_dir, params.stack_version_buildnum) File "/var/lib/ambari-agent/cache/common-services/NIFI_REGISTRY/0.1.0/package/scripts/nifi_toolkit_util.py", line 342, in encrypt_sensitive_properties encrypt_config_script = get_toolkit_script('encrypt-config.sh',toolkit_tmp_dir, stack_version_buildnum) File "/var/lib/ambari-agent/cache/common-services/NIFI_REGISTRY/0.1.0/package/scripts/nifi_toolkit_util.py", line 61, in get_toolkit_script raise Exception("Couldn't find nifi toolkit directory in " + scriptDir) Exception: Couldn't find nifi toolkit directory in /var/lib/ambari-agent/tmp stdout: 2018-08-20 16:11:12,671 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.1.0-35 -> 3.1.1.0-35 2018-08-20 16:11:12,849 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.1.0-35 -> 3.1.1.0-35 2018-08-20 16:11:12,852 - Group['nifiregistry'] {} 2018-08-20 16:11:12,853 - Group['hadoop'] {} 2018-08-20 16:11:12,853 - Group['nifi'] {} 2018-08-20 16:11:12,854 - User['streamline'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,855 - User['logsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,855 - User['registry'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,856 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,857 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,857 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,858 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,859 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,860 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-08-20 16:11:12,860 - User['nifiregistry'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['nifiregistry'], 'uid': None} 2018-08-20 16:11:12,861 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['nifi'], 'uid': None} 2018-08-20 16:11:12,862 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2018-08-20 16:11:12,863 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'} 2018-08-20 16:11:12,868 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if 2018-08-20 16:11:12,883 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'} 2018-08-20 16:11:12,889 - Skipping Execute[('setenforce', '0')] due to not_if 2018-08-20 16:11:12,889 - Testing the JVM's JCE policy to see it if supports an unlimited key length. 2018-08-20 16:11:13,068 - The unlimited key JCE policy is required, and appears to have been installed. 2018-08-20 16:11:13,313 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.1.0-35 -> 3.1.1.0-35 2018-08-20 16:11:13,325 - Toolkit files dir is /var/lib/ambari-agent/cache/common-services/NIFI_REGISTRY/0.1.0/package/files 2018-08-20 16:11:13,325 - Toolkit tmp dir is /var/lib/ambari-agent/tmp 2018-08-20 16:11:13,376 - Directory['/var/lib/ambari-agent/tmp/nifi-toolkit-1.7.0.3.2.0.0-520'] {'group': 'nifiregistry', 'cd_access': 'a', 'create_parents': False, 'recursive_ownership': True, 'owner': 'nifiregistry', 'mode': 0755} 2018-08-20 16:11:13,378 - Changing owner for /var/lib/ambari-agent/tmp/nifi-toolkit-1.7.0.3.2.0.0-520 from 0 to nifiregistry 2018-08-20 16:11:13,378 - Changing group for /var/lib/ambari-agent/tmp/nifi-toolkit-1.7.0.3.2.0.0-520 from 0 to nifiregistry 2018-08-20 16:11:13,388 - Directory['/var/run/nifi-registry'] {'owner': 'nifiregistry', 'create_parents': True, 'group': 'nifiregistry', 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,388 - Directory['/var/lib/nifi-registry'] {'owner': 'nifiregistry', 'create_parents': True, 'group': 'nifiregistry', 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,389 - Directory['/var/lib/nifi-registry/conf'] {'owner': 'nifiregistry', 'create_parents': True, 'group': 'nifiregistry', 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,390 - Directory['/var/lib/nifi-registry/database'] {'owner': 'nifiregistry', 'create_parents': True, 'group': 'nifiregistry', 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,391 - Directory['/usr/hdf/current/nifi-registry/conf'] {'owner': 'nifiregistry', 'create_parents': True, 'group': 'nifiregistry', 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,391 - Directory['/usr/hdf/current/nifi-registry/bin'] {'owner': 'nifiregistry', 'create_parents': True, 'group': 'nifiregistry', 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,392 - Directory['/usr/hdf/current/nifi-registry/lib'] {'owner': 'nifiregistry', 'create_parents': True, 'group': 'nifiregistry', 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,394 - Directory['/usr/hdf/current/nifi-registry/docs'] {'owner': 'nifiregistry', 'group': 'nifiregistry', 'create_parents': True, 'recursive_ownership': True, 'cd_access': 'a'} 2018-08-20 16:11:13,396 - PropertiesFile['/usr/hdf/current/nifi-registry/conf/nifi-registry.properties'] {'owner': 'nifiregistry', 'group': 'nifiregistry', 'mode': 0600, 'properties': ...} 2018-08-20 16:11:13,402 - Generating properties file: /usr/hdf/current/nifi-registry/conf/nifi-registry.properties 2018-08-20 16:11:13,402 - File['/usr/hdf/current/nifi-registry/conf/nifi-registry.properties'] {'owner': 'nifiregistry', 'content': InlineTemplate(...), 'group': 'nifiregistry', 'mode': 0600, 'encoding': 'UTF-8'} 2018-08-20 16:11:13,422 - Writing File['/usr/hdf/current/nifi-registry/conf/nifi-registry.properties'] because contents don't match 2018-08-20 16:11:13,425 - File['/usr/hdf/current/nifi-registry/conf/bootstrap.conf'] {'owner': 'nifiregistry', 'content': InlineTemplate(...), 'group': 'nifiregistry', 'mode': 0600} 2018-08-20 16:11:13,428 - File['/usr/hdf/current/nifi-registry/conf/logback.xml'] {'owner': 'nifiregistry', 'content': InlineTemplate(...), 'group': 'nifiregistry', 'mode': 0400} 2018-08-20 16:11:13,436 - File['/usr/hdf/current/nifi-registry/conf/authorizers.xml'] {'owner': 'nifiregistry', 'content': InlineTemplate(...), 'group': 'nifiregistry', 'mode': 0600} 2018-08-20 16:11:13,440 - File['/usr/hdf/current/nifi-registry/conf/identity-providers.xml'] {'owner': 'nifiregistry', 'content': InlineTemplate(...), 'group': 'nifiregistry', 'mode': 0600} 2018-08-20 16:11:13,442 - File['/usr/hdf/current/nifi-registry/conf/providers.xml'] {'owner': 'nifiregistry', 'content': InlineTemplate(...), 'group': 'nifiregistry', 'mode': 0400} 2018-08-20 16:11:13,444 - File['/usr/hdf/current/nifi-registry/bin/nifi-registry-env.sh'] {'owner': 'nifiregistry', 'content': InlineTemplate(...), 'group': 'nifiregistry', 'mode': 0755} 2018-08-20 16:11:13,445 - Encrypting NiFi Registry sensitive configuration properties Command failed after 1 tries
Created 08-20-2018 04:42 PM
@Nick Lewis looks like it is failing with "Exception: Couldn't find nifi toolkit directory in /var/lib/ambari-agent/tmp stdou" error.
Can you verify with https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.2.0/ambari-managed-hdf-upgrade/content/hdf-veri...
Created 08-20-2018 04:42 PM
@Nick Lewis looks like it is failing with "Exception: Couldn't find nifi toolkit directory in /var/lib/ambari-agent/tmp stdou" error.
Can you verify with https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.2.0/ambari-managed-hdf-upgrade/content/hdf-veri...
Created 08-20-2018 06:21 PM
That solved my problem! I went to Hortonworks downloads and grabbed the correct versions of Nifi toolkit. I am now onto the next error:
resource_management.core.exceptions.ExecutionFailed: Execution of '/var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.1.0-35/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -f /var/lib/nifi/conf/flow.xml.gz -s '[PROTECTED]' -a /usr/hdf/current/nifi/conf/authorizers.xml -p '[PROTECTED]'' returned 1. Error: Could not find or load main class org.apache.nifi.toolkit.encryptconfig.EncryptConfigMain
But I'll save that for another post once I've investigated some more, thanks!