Support Questions
Find answers, ask questions, and share your expertise

Install Nifi 3.0 on HDP 2.6.3 Getting "resource_management.core.exceptions.Fail: Configuration parameter 'availableServices' was not found in configurations dictionary!"

New Contributor

I'm trying to install HDF 3.0 onto an existing HDP cluster. I followed the directions here https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1.1/bk_installing-hdf-and-hdp/content/ch_add-... and was able to install storm, kafka, registry, and sam with no problem. However installing nifi fails. It's obvious that there's some configuration information missing somewhere, but I don't know where. The Nifi certificate authority does install.

This is stderr:
Traceback (most recent call last):

  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 309, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 71, in install
    self.configure(env, True)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 120, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 112, in configure
    PropertiesFile(params.nifi_config_dir + '/nifi.properties', properties = params.nifi_properties, mode = 0600, owner = params.nifi_user, group = params.nifi_group)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/properties_file.py", line 54, in action_create
    mode = self.resource.mode
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 123, in action_create
    content = self._get_content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 160, in _get_content
    return content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 52, in __call__
    return self.get_content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 144, in get_content
    rendered = self.template.render(self.context)
  File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 144, in get_content
    rendered = self.template.render(self.context)
  File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__
    raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'availableServices' was not found in configurations dictionary!

stdout (I truncated all but the last few lines of stdout):


2017-11-02 13:11:16,171 - Directory['/var/lib/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-11-02 13:11:16,171 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-11-02 13:11:16,173 - Directory['{{nifi_content_repo_dir_default}}'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-11-02 13:11:16,174 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True}
2017-11-02 13:11:16,174 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2017-11-02 13:11:16,180 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2017-11-02 13:11:16,181 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2017-11-02 13:11:16,188 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2017-11-02 13:11:16,188 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-11-02 13:11:16,290 - The repository with version 2.6.3.0-235 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2017-11-02 13:11:16,291 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure.

Command failed after 1 tries
16 REPLIES 16

New Contributor

Not an answer, but another comment. I'm not sure that this is a cause or effect, but I've found that /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0 has no configuration directory nor a metainfo.xml file, just a package directory (even on the ambari server ndoe.) Nifi is installed on the node and can be run as a service. It just isn't able to be controlled by ambari.

Rising Star

Hi Raymond, what version of Ambari is your cluster running?

New Contributor

Should be the version that's in 2.6.3, but this is the result of the rpm query:

ambari-server-2.6.0.0-267.x86_64

Rising Star

The problem you experienced is because the current version of HDF 3.0.1.1 is only compatible with Ambari 2.5.1 (see the Component Availability Matrix here https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1.1/bk_release-notes/content/ch_hdf_relnotes..... There is an upcoming release of HDF that will address this problem for 2.6.0.

New Contributor

Okay, interesting as SAM and Registry work fine. Thanks.

Rising Star

Yes the NiFi service has a reliance on an Ambari configuration that is no longer available in Ambari 2.6.0. The upcoming release removes that dependency.

Cloudera Employee

@Yolanda, I Ran into the same issue just now :). Seems like have to downgrade to Ambari 2.5.1 and re-install is the only option available?

Explorer

Hi @Muki Soomar,

First make sure that you uninstall the previous mpack (HDF 3.0.1.1), next install the new mpack 3.0.2.

It should work after that.

Regards,

New Contributor

@Soomar, as I said, you can just install the deployed nifi software and run it as a service. The only thing ATM that you miss out on is starting/stoping from Ambari and changing the configuration from it. Clustering is _much_ easier if you have it running from Ambari also.