Support Questions
Find answers, ask questions, and share your expertise

Install Nifi 3.0 on HDP 2.6.3 Getting "resource_management.core.exceptions.Fail: Configuration parameter 'availableServices' was not found in configurations dictionary!"

New Contributor

I'm trying to install HDF 3.0 onto an existing HDP cluster. I followed the directions here https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1.1/bk_installing-hdf-and-hdp/content/ch_add-... and was able to install storm, kafka, registry, and sam with no problem. However installing nifi fails. It's obvious that there's some configuration information missing somewhere, but I don't know where. The Nifi certificate authority does install.

This is stderr:
Traceback (most recent call last):

  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 309, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 71, in install
    self.configure(env, True)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 120, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 112, in configure
    PropertiesFile(params.nifi_config_dir + '/nifi.properties', properties = params.nifi_properties, mode = 0600, owner = params.nifi_user, group = params.nifi_group)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/properties_file.py", line 54, in action_create
    mode = self.resource.mode
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 123, in action_create
    content = self._get_content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 160, in _get_content
    return content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 52, in __call__
    return self.get_content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 144, in get_content
    rendered = self.template.render(self.context)
  File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 144, in get_content
    rendered = self.template.render(self.context)
  File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__
    raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'availableServices' was not found in configurations dictionary!

stdout (I truncated all but the last few lines of stdout):


2017-11-02 13:11:16,171 - Directory['/var/lib/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-11-02 13:11:16,171 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-11-02 13:11:16,173 - Directory['{{nifi_content_repo_dir_default}}'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-11-02 13:11:16,174 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True}
2017-11-02 13:11:16,174 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2017-11-02 13:11:16,180 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2017-11-02 13:11:16,181 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2017-11-02 13:11:16,188 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2017-11-02 13:11:16,188 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-11-02 13:11:16,290 - The repository with version 2.6.3.0-235 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2017-11-02 13:11:16,291 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure.

Command failed after 1 tries
16 REPLIES 16

Rising Star

Yes that is correct, I apologize for not being clear. The Ambari code in the HDF management pack that manages the NiFi service has a reliance on an Ambari configuration that is no longer available in Ambari 2.6.0.

Cloudera Employee

@Raymond Pfaff and @Yolanda M. Davis. Yes. Thanks, I just saw that.

Cloudera Employee

Hi @Raymond Pfaff

Ambari MPACKs are only compatible with the version of Ambari they were written against. You can then only upgrade to the next release of each product that coincides Ambari support ( ex. HDP 2.6.1 and HDF 3.0 against Ambari 2.5.1.0 )

HDF 3.0.2 has just been released and supports Ambari 2.6.0 :
https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.2/bk_support-matrices/content/ch_matrices-hdf...

That should take care of the prb.

Regards,

Cloudera Employee

Hi @Raymond Pfaff

Ambari MPACKs are only compatible with the version of Ambari they were written against.

You can only upgrade to the next release of each product that coincides with Ambari support ( ex. HDP 2.6.1 and HDF 3.0 against Ambari 2.5.1.0 )

HDF 3.0.2 has just been released and supports Ambari 2.6.0 :
https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.2/bk_support-matrices/content/ch_matrices-hdf...

That should take care of the prb.

Regards,

New Contributor

I am using HDP-2.6.5.0 and HDF-ambari-mpack-3.1.2.0-7 still NIFI is not working showing error

Skipping stack-select on NIFI because it does not exist in the stack-select package structure.

New Contributor

I am also using HDP-2.6.5.0 and HDF-ambari-mpack-3.1.2.0-7 and am facing the same issue

  1. Skipping stack-select on NIFI because it does not exist in the stack-selectpackage structure.

Explorer

@Ashish Kumar ,@Misbah Rehman : Were you able to resolve the issue?