Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ERROR on Installing NIFI - as new service in HDP v2.6

Highlighted

ERROR on Installing NIFI - as new service in HDP v2.6

New Contributor

Hello, when trying to install the NIFI service in HDP v2.6 via Ambari, I get the following error - and service is NOT installed:

stderr: /var/lib/ambari-agent/data/errors-279.txt
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 131, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 50, in install
    Execute('tar -xf '+params.temp_file+' -C '+ params.nifi_dir +' >> ' + params.nifi_log_file, user=params.nifi_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'tar -xf /tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -C /opt/nifi-1.1.0.2.1.2.0-10-bin >> /var/log/nifi/nifi-setup.log' returned 2. gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
6 REPLIES 6

Re: ERROR on Installing NIFI - as new service in HDP v2.6

Super Mentor

@Leonard Edwin

Looks like the "/tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz" file is corrupted. Can you please try to extract that file manually to see if the nifi tar.gz is correct?

Can you retry installing? Is there any intermittent internet issue while downloading the nifi packages?

Re: ERROR on Installing NIFI - as new service in HDP v2.6

New Contributor

@sensharma, Jay Kumar SenSharma, I am experiencing the same problem. I couldn't extract the file nifi-1.1.0.2.1.2.0-10-bin.tar.gz manually. Any help please ? Thanks so much ! I appreciate it !

Re: ERROR on Installing NIFI - as new service in HDP v2.6

New Contributor

@Leonard Edwin, did you resolve this problem ? I had experienced the same error. Thanks !

Re: ERROR on Installing NIFI - as new service in HDP v2.6

New Contributor

Facing same issue , checking nifi install log under /var/log/nifi/nifi-setup.log

https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz
Resolving public-repo-1.hortonworks.com... 54.230.44.44, 54.230.44.35, 54.230.44.173, ...
Connecting to public-repo-1.hortonworks.com|54.230.44.44|:443... connected.
HTTP request sent, awaiting response... 404 Not Found

Re: ERROR on Installing NIFI - as new service in HDP v2.6

New Contributor

to solve 404 , download file from apache site

replace

wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz

with

wget https://archive.apache.org/dist/nifi/1.1.0/nifi-toolkit-1.1.0-bin.tar.gz

and proceed with configurtion

// isseue configuring nifi through Ambari

stderr: /var/lib/ambari-agent/data/errors-783.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 131, in <module> Master().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 40, in install Execute('wget '+params.snapshot_package+' -O '+params.temp_file+' -a ' + params.nifi_log_file, user=params.nifi_user) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -O /tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -a /var/log/nifi/nifi-setup.log' returned 8. stdout: /var/lib/ambari-agent/data/output-783.txt 2018-03-13 06:51:53,284 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=None -> 2.6 2018-03-13 06:51:53,285 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf 2018-03-13 06:51:53,287 - Group['livy'] {} 2018-03-13 06:51:53,289 - Group['spark'] {} 2018-03-13 06:51:53,289 - Group['ranger'] {} 2018-03-13 06:51:53,289 - Group['hdfs'] {} 2018-03-13 06:51:53,289 - Group['zeppelin'] {} 2018-03-13 06:51:53,290 - Group['hadoop'] {} 2018-03-13 06:51:53,290 - Group['nifi'] {} 2018-03-13 06:51:53,290 - Adding group Group['nifi'] 2018-03-13 06:51:53,476 - Group['users'] {} 2018-03-13 06:51:53,477 - Group['knox'] {} 2018-03-13 06:51:53,477 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,481 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,485 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,489 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,492 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,492 - User['atlas'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,493 - User['falcon'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,494 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['ranger'], 'uid': None} 2018-03-13 06:51:53,495 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,496 - User['zeppelin'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['zeppelin', 'hadoop'], 'uid': None} 2018-03-13 06:51:53,497 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,498 - Adding user User['nifi'] 2018-03-13 06:51:53,637 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,639 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,640 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,641 - User['flume'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,642 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,643 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs'], 'uid': None} 2018-03-13 06:51:53,644 - User['sqoop'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,645 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,646 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,647 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,648 - User['knox'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,649 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,650 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2018-03-13 06:51:53,655 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'} 2018-03-13 06:51:53,729 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if 2018-03-13 06:51:53,730 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'} 2018-03-13 06:51:53,731 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2018-03-13 06:51:53,733 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2018-03-13 06:51:53,735 - call['/var/lib/ambari-agent/tmp/changeUid.sh hbase'] {} 2018-03-13 06:51:53,812 - call returned (0, '1002') 2018-03-13 06:51:53,812 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1002'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'} 2018-03-13 06:51:53,893 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1002'] due to not_if 2018-03-13 06:51:53,893 - Group['hdfs'] {} 2018-03-13 06:51:53,894 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hdfs']} 2018-03-13 06:51:53,894 - FS Type: 2018-03-13 06:51:53,894 - Directory['/etc/hadoop'] {'mode': 0755} 2018-03-13 06:51:53,924 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2018-03-13 06:51:53,925 - Writing File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] because contents don't match 2018-03-13 06:51:53,927 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777} 2018-03-13 06:51:53,951 - Repository['HDP-2.6-repo-1'] {'append_to_file': False, 'base_url': 'http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.3.0', 'action': ['create'], 'components': ['HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None} 2018-03-13 06:51:53,962 - File['/etc/yum.repos.d/ambari-hdp-1.repo'] {'content': '[HDP-2.6-repo-1]\nname=HDP-2.6-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.3.0\n\npath=/\nenabled=1\ngpgcheck=0'} 2018-03-13 06:51:53,963 - Writing File['/etc/yum.repos.d/ambari-hdp-1.repo'] because contents don't match 2018-03-13 06:51:53,964 - Repository['HDP-UTILS-1.1.0.21-repo-1'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6', 'action': ['create'], 'components': ['HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None} 2018-03-13 06:51:53,973 - File['/etc/yum.repos.d/ambari-hdp-1.repo'] {'content': '[HDP-2.6-repo-1]\nname=HDP-2.6-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.3.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-UTILS-1.1.0.21-repo-1]\nname=HDP-UTILS-1.1.0.21-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6\n\npath=/\nenabled=1\ngpgcheck=0'} 2018-03-13 06:51:53,974 - Writing File['/etc/yum.repos.d/ambari-hdp-1.repo'] because contents don't match 2018-03-13 06:51:53,975 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2018-03-13 06:51:54,237 - Skipping installation of existing package unzip 2018-03-13 06:51:54,238 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2018-03-13 06:51:54,341 - Skipping installation of existing package curl 2018-03-13 06:51:54,342 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2018-03-13 06:51:54,466 - Skipping installation of existing package hdp-select 2018-03-13 06:51:54,469 - The repository with version 2.6.3.0-235 for this command has been marked as resolved. It will be used to report the version of the component which was installed 2018-03-13 06:51:54,470 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure. 2018-03-13 06:51:54,858 - Directory['/var/run/nifi'] {'owner': 'nifi', 'group': 'nifi'} 2018-03-13 06:51:54,859 - Creating directory Directory['/var/run/nifi'] since it doesn't exist. 2018-03-13 06:51:54,860 - Changing owner for /var/run/nifi from 0 to nifi 2018-03-13 06:51:54,860 - Changing group for /var/run/nifi from 0 to nifi 2018-03-13 06:51:54,860 - Directory['/var/log/nifi'] {'owner': 'nifi', 'group': 'nifi'} 2018-03-13 06:51:54,861 - Creating directory Directory['/var/log/nifi'] since it doesn't exist. 2018-03-13 06:51:54,861 - Changing owner for /var/log/nifi from 0 to nifi 2018-03-13 06:51:54,861 - Changing group for /var/log/nifi from 0 to nifi 2018-03-13 06:51:54,861 - Execute['touch /var/log/nifi/nifi-setup.log'] {'user': 'nifi'} 2018-03-13 06:51:55,001 - Execute['wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -O /tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -a /var/log/nifi/nifi-setup.log'] {'user': 'nifi'} 2018-03-13 06:51:56,472 - The repository with version 2.6.3.0-235 for this command has been marked as resolved. It will be used to report the version of the component which was installed 2018-03-13 06:51:56,479 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure. Command failed after 1 tries

Re: ERROR on Installing NIFI - as new service in HDP v2.6

New Contributor

The link is broken. If you try to execute wget on command line, you get

# wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz
--2018-04-06 13:07:00--  https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz
Resolving public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)... 13.32.14.146, 13.32.14.230, 13.32.14.164, ...
Connecting to public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)|13.32.14.146|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-04-06 13:07:01 ERROR 404: Not Found.

And if you open URL using browser

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz</Key>
<RequestId>2530B9077A50BDCE</RequestId>
<HostId>
2JMQs01QCEreVQu039RRS7KoSyN8u5lUSXvZ6AFuAJwgrwDBIYRqRWOX70aVQEMv5IcH1T6JCZI=
</HostId>
</Error>
Don't have an account?
Coming from Hortonworks? Activate your account here