Created 05-02-2017 10:39 PM
Hello, when trying to install the NIFI service in HDP v2.6 via Ambari, I get the following error - and service is NOT installed:
stderr: /var/lib/ambari-agent/data/errors-279.txtTraceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 131, in <module> Master().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 50, in install Execute('tar -xf '+params.temp_file+' -C '+ params.nifi_dir +' >> ' + params.nifi_log_file, user=params.nifi_user) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'tar -xf /tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -C /opt/nifi-1.1.0.2.1.2.0-10-bin >> /var/log/nifi/nifi-setup.log' returned 2. gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now
Created 05-03-2017 12:25 AM
Looks like the "/tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz" file is corrupted. Can you please try to extract that file manually to see if the nifi tar.gz is correct?
Can you retry installing? Is there any intermittent internet issue while downloading the nifi packages?
Created 12-12-2017 02:27 PM
@sensharma, Jay Kumar SenSharma, I am experiencing the same problem. I couldn't extract the file nifi-1.1.0.2.1.2.0-10-bin.tar.gz manually. Any help please ? Thanks so much ! I appreciate it !
Created 12-12-2017 02:21 PM
@Leonard Edwin, did you resolve this problem ? I had experienced the same error. Thanks !
Created 02-03-2018 09:00 PM
Facing same issue , checking nifi install log under /var/log/nifi/nifi-setup.log
https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz Resolving public-repo-1.hortonworks.com... 54.230.44.44, 54.230.44.35, 54.230.44.173, ... Connecting to public-repo-1.hortonworks.com|54.230.44.44|:443... connected. HTTP request sent, awaiting response... 404 Not Found
Created 03-13-2018 01:11 PM
to solve 404 , download file from apache site
replace
wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz
with
wget https://archive.apache.org/dist/nifi/1.1.0/nifi-toolkit-1.1.0-bin.tar.gz
and proceed with configurtion
// isseue configuring nifi through Ambari
stderr: /var/lib/ambari-agent/data/errors-783.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 131, in <module> Master().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.6/services/NIFI/package/scripts/master.py", line 40, in install Execute('wget '+params.snapshot_package+' -O '+params.temp_file+' -a ' + params.nifi_log_file, user=params.nifi_user) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -O /tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -a /var/log/nifi/nifi-setup.log' returned 8. stdout: /var/lib/ambari-agent/data/output-783.txt 2018-03-13 06:51:53,284 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=None -> 2.6 2018-03-13 06:51:53,285 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf 2018-03-13 06:51:53,287 - Group['livy'] {} 2018-03-13 06:51:53,289 - Group['spark'] {} 2018-03-13 06:51:53,289 - Group['ranger'] {} 2018-03-13 06:51:53,289 - Group['hdfs'] {} 2018-03-13 06:51:53,289 - Group['zeppelin'] {} 2018-03-13 06:51:53,290 - Group['hadoop'] {} 2018-03-13 06:51:53,290 - Group['nifi'] {} 2018-03-13 06:51:53,290 - Adding group Group['nifi'] 2018-03-13 06:51:53,476 - Group['users'] {} 2018-03-13 06:51:53,477 - Group['knox'] {} 2018-03-13 06:51:53,477 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,481 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,485 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,489 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,492 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,492 - User['atlas'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,493 - User['falcon'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,494 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['ranger'], 'uid': None} 2018-03-13 06:51:53,495 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,496 - User['zeppelin'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['zeppelin', 'hadoop'], 'uid': None} 2018-03-13 06:51:53,497 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,498 - Adding user User['nifi'] 2018-03-13 06:51:53,637 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,639 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,640 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None} 2018-03-13 06:51:53,641 - User['flume'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,642 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,643 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs'], 'uid': None} 2018-03-13 06:51:53,644 - User['sqoop'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,645 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,646 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,647 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,648 - User['knox'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,649 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2018-03-13 06:51:53,650 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2018-03-13 06:51:53,655 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'} 2018-03-13 06:51:53,729 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if 2018-03-13 06:51:53,730 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'} 2018-03-13 06:51:53,731 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2018-03-13 06:51:53,733 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2018-03-13 06:51:53,735 - call['/var/lib/ambari-agent/tmp/changeUid.sh hbase'] {} 2018-03-13 06:51:53,812 - call returned (0, '1002') 2018-03-13 06:51:53,812 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1002'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'} 2018-03-13 06:51:53,893 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1002'] due to not_if 2018-03-13 06:51:53,893 - Group['hdfs'] {} 2018-03-13 06:51:53,894 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hdfs']} 2018-03-13 06:51:53,894 - FS Type: 2018-03-13 06:51:53,894 - Directory['/etc/hadoop'] {'mode': 0755} 2018-03-13 06:51:53,924 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2018-03-13 06:51:53,925 - Writing File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] because contents don't match 2018-03-13 06:51:53,927 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777} 2018-03-13 06:51:53,951 - Repository['HDP-2.6-repo-1'] {'append_to_file': False, 'base_url': 'http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.3.0', 'action': ['create'], 'components': ['HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None} 2018-03-13 06:51:53,962 - File['/etc/yum.repos.d/ambari-hdp-1.repo'] {'content': '[HDP-2.6-repo-1]\nname=HDP-2.6-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.3.0\n\npath=/\nenabled=1\ngpgcheck=0'} 2018-03-13 06:51:53,963 - Writing File['/etc/yum.repos.d/ambari-hdp-1.repo'] because contents don't match 2018-03-13 06:51:53,964 - Repository['HDP-UTILS-1.1.0.21-repo-1'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6', 'action': ['create'], 'components': ['HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None} 2018-03-13 06:51:53,973 - File['/etc/yum.repos.d/ambari-hdp-1.repo'] {'content': '[HDP-2.6-repo-1]\nname=HDP-2.6-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.3.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-UTILS-1.1.0.21-repo-1]\nname=HDP-UTILS-1.1.0.21-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6\n\npath=/\nenabled=1\ngpgcheck=0'} 2018-03-13 06:51:53,974 - Writing File['/etc/yum.repos.d/ambari-hdp-1.repo'] because contents don't match 2018-03-13 06:51:53,975 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2018-03-13 06:51:54,237 - Skipping installation of existing package unzip 2018-03-13 06:51:54,238 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2018-03-13 06:51:54,341 - Skipping installation of existing package curl 2018-03-13 06:51:54,342 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2018-03-13 06:51:54,466 - Skipping installation of existing package hdp-select 2018-03-13 06:51:54,469 - The repository with version 2.6.3.0-235 for this command has been marked as resolved. It will be used to report the version of the component which was installed 2018-03-13 06:51:54,470 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure. 2018-03-13 06:51:54,858 - Directory['/var/run/nifi'] {'owner': 'nifi', 'group': 'nifi'} 2018-03-13 06:51:54,859 - Creating directory Directory['/var/run/nifi'] since it doesn't exist. 2018-03-13 06:51:54,860 - Changing owner for /var/run/nifi from 0 to nifi 2018-03-13 06:51:54,860 - Changing group for /var/run/nifi from 0 to nifi 2018-03-13 06:51:54,860 - Directory['/var/log/nifi'] {'owner': 'nifi', 'group': 'nifi'} 2018-03-13 06:51:54,861 - Creating directory Directory['/var/log/nifi'] since it doesn't exist. 2018-03-13 06:51:54,861 - Changing owner for /var/log/nifi from 0 to nifi 2018-03-13 06:51:54,861 - Changing group for /var/log/nifi from 0 to nifi 2018-03-13 06:51:54,861 - Execute['touch /var/log/nifi/nifi-setup.log'] {'user': 'nifi'} 2018-03-13 06:51:55,001 - Execute['wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -O /tmp/nifi-1.1.0.2.1.2.0-10-bin.tar.gz -a /var/log/nifi/nifi-setup.log'] {'user': 'nifi'} 2018-03-13 06:51:56,472 - The repository with version 2.6.3.0-235 for this command has been marked as resolved. It will be used to report the version of the component which was installed 2018-03-13 06:51:56,479 - Skipping stack-select on NIFI because it does not exist in the stack-select package structure. Command failed after 1 tries
Created 04-06-2018 07:39 PM
The link is broken. If you try to execute wget on command line, you get
# wget https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz --2018-04-06 13:07:00-- https://public-repo-1.hortonworks.com/HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz Resolving public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)... 13.32.14.146, 13.32.14.230, 13.32.14.164, ... Connecting to public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)|13.32.14.146|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-04-06 13:07:01 ERROR 404: Not Found.
And if you open URL using browser
This XML file does not appear to have any style information associated with it. The document tree is shown below. <Error> <Code>NoSuchKey</Code> <Message>The specified key does not exist.</Message> <Key>HDF/2.1.2.0/nifi-1.1.0.2.1.2.0-10-bin.tar.gz</Key> <RequestId>2530B9077A50BDCE</RequestId> <HostId> 2JMQs01QCEreVQu039RRS7KoSyN8u5lUSXvZ6AFuAJwgrwDBIYRqRWOX70aVQEMv5IcH1T6JCZI= </HostId> </Error>