I followed the instructions on this page to set HDF up on an EC2 instance: https://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.1.2/bk_dataflow-ambari-installation/content/dow...
I got everything configured and started but now NIFI isn't showing up as a service I can add through Ambari. I am thinking I missed the install HDF management pack step. When I go here, I don't see an install for redhat 7: https://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.1.2/bk_dataflow-ambari-installation/content/ins... Where can I find this and will it work for me to install it now after going through everything? Or is there a separate way to install NIFI?
Without installing the HDF mpack into your Ambari server you will not really have HDF, but instead will have the default HDP stack.
For Redhat 7 just use the CENTOS 7 mpack directions:
ambari-server install-mpack \ --mpack=http://public-repo-1.hortonworks.com/HDF/centos7/2.x/updates/184.108.40.206/tars/hdf_ambari_mp/hdf-ambari-mpack-220.127.116.11-10.tar.gz \ --purge \ --verbose
Not sure hat you mean by installing everything? You are saying you ran completely through the Ambari install wizard and installed what? Sounds like you installed HDP stack version components. They are likely not going to be the same versions of things in the HDF stack.
You will need to run through the wizard again after installing the HDF mpack.
# ambari-server stop # ambari-server reset # ambari-server install-mpack --mpack=http://public-repo-1.hortonworks.com/HDF/centos7/2.x/updates/18.104.22.168/tars/hdf_ambari_mp/hdf-ambari-mpack-22.214.171.124-10.tar.gz --purge --verbose # ambari-server start
Access the Ambari UI and run through the wizard again. This time you should see all the HDF component services including NiFi.
I followed your suggestions. Stop & reset Ambari server, install the mpack, then when I went to restart Ambari server, it wouldn't start. The log has the following in it:
here is more of the error I was able to find: 28 Apr 2017 08:48:17,171 ERROR [main] AmbariServer:1017 - Failed to run the Ambari Server com.google.inject.ProvisionException: Guice provision errors: 1) Error injecting constructor, org.apache.ambari.server.AmbariException: Unable to parse stack upgrade file at location: /var/lib/ambari-server/resources/stacks/HDF/2.0/upgrades/nonrolling-upgrade-2.0.xml at org.apache.ambari.server.stack.StackManager.<init>(StackManager.java:144) while locating org.apache.ambari.server.stack.StackManager annotated with interface com.google.inject.assistedinject.Assisted at org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:261) at org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:135) while locating org.apache.ambari.server.api.services.AmbariMetaInfo for field at org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:166) at org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:166) while locating org.apache.ambari.server.controller.AmbariServer 1 error
I actually started all over with a new VM, ran through the install steps and I get the same error message when trying to start Ambari server.
I noticed there is a nonrolling-upgrade2.0xml and a nonrolling-upgrade.2.1.xml file in the location above. Is that maybe what is hanging this up?
Any suggestions on what I should try next? I didn't find too much this message on google.
Thanks for the help! John
@Matt Clarke, I did run it with the --purge option. I copied it exactly has you had it above and it still gave the same error message. I just ran it again this morning just to be sure. After the prompt to make sure I want to do the purge, I get the following:
CAUTION: You have specified the --purge option with --purge-list=['stack-definitions', 'mpacks']. This will replace all existing stack definitions, management packs currently installed. Are you absolutely sure you want to perform the purge [yes/no]? (no)y INFO: Loading properties from /etc/ambari-server/conf/ambari.properties INFO: Purging existing stack definitions and management packs INFO: Purging stack location: /var/lib/ambari-server/resources/stacks INFO: Purging mpacks staging location: /var/lib/ambari-server/resources/mpacks INFO: Loading properties from /etc/ambari-server/conf/ambari.properties INFO: Stage management pack hdf-ambari-mpack-126.96.36.199-10 to staging location /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-188.8.131.52-10 INFO: Processing artifact hdf-service-definitions of type service-definitions in /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-184.108.40.206-10/common-services INFO: Loading properties from /etc/ambari-server/conf/ambari.properties Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 941, in <module> mainBody() File "/usr/sbin/ambari-server.py", line 911, in mainBody main(options, args, parser) File "/usr/sbin/ambari-server.py", line 863, in main action_obj.execute() File "/usr/sbin/ambari-server.py", line 78, in execute self.fn(*self.args, **self.kwargs) File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 861, in install_mpack (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, mpack_archive_path) = _install_mpack(options, replay_mode) File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 777, in _install_mpack process_service_definitions_artifact(artifact, artifact_source_dir, options) File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 514, in process_service_definitions_artifact create_symlink(src_service_definitions_dir, dest_service_definitions_dir, file, options.force) File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 235, in create_symlink create_symlink_using_path(src_path, dest_link, force) File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 246, in create_symlink_using_path sudo.symlink(src_path, dest_link) File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", line 93, in symlink os.symlink(source, link_name) OSError: [Errno 17] File exists
Does this point to another issue or maybe the root of my server start up issue?
@John Daniels I haven't seen a problem like this personally but one thing I'd suggest is to ensure that the permissions on that file allow the ambari user to read the file. The other thing I'd try is actually reading that file (which is text/xml based) to make sure it's not corrupted in any way.
@Yolanda M. Davis, I ran the following to make sure it wasn't a permissions issue but I get the same error:
sudo chmod 777 /var/lib/ambari-server/resources/stacks/HDF/2.0/upgrades/nonrolling-upgrade-2.0.xml
I also was able to open this file in VI so I would assume its not corrupted but that may be a bad assumption.