Member since
08-10-2018
6
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2077 | 08-13-2018 05:20 PM |
05-09-2019
08:42 PM
Works like a charm!!
... View more
04-30-2019
04:19 AM
@ zhoussen , the link for "Processing Notifications in real-time with Apache NiFi " - flow below is broken. Can you please add this?
... View more
09-05-2018
04:00 PM
@Greg Keys, The solution you provided for the table definition and file format worked for me? But any insights why we need to have it as part of the data load?
... View more
08-13-2018
05:25 PM
Please add "yum remove hdp-select* -y" and "rm -rf /usr/bin/oozie-client".. I found this was missing when i was uninstalling it using the above script
... View more
08-13-2018
05:20 PM
Hi @Akhil, Thank you for the response. Yes, this answer was really helpful. One thing, I wanted to add was there was hdp-select already installed with the same version still it was failing. Once, I uninstalled hdp-select, the ambari installed the fresh one and it started working. Thank you Once again!!
... View more
08-12-2018
02:58 AM
Hi Geeks, I am trying to set up the HortonWorks cluster using ambari 2.7.0.0 and HDP 3.0.0.0 and using public repositories. But it is keep failing at the Install, start, test step with the below errors. No one of the components are getting installed. stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stack-hooks/before-INSTALL/scripts/hook.py", line 37, in <module>
BeforeInstallHook().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 363, in execute
self.save_component_version_to_structured_out(self.command_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 222, in save_component_version_to_structured_out
stack_select_package_name = stack_select.get_package_name()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 109, in get_package_name
package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 223, in get_packages
supported_packages = get_supported_packages()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 147, in get_supported_packages
raise Fail("Unable to query for supported packages using {0}".format(stack_selector_path))
resource_management.core.exceptions.Fail: Unable to query for supported packages using /usr/bin/hdp-select
stdout:
2018-08-10 16:30:56,129 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=None -> 3.0
2018-08-10 16:30:56,134 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2018-08-10 16:30:56,135 - Group['hdfs'] {}
2018-08-10 16:30:56,136 - Group['hadoop'] {}
2018-08-10 16:30:56,136 - Group['users'] {}
2018-08-10 16:30:56,136 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-10 16:30:56,138 - User['yarn-ats'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-10 16:30:56,139 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-10 16:30:56,140 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-10 16:30:56,141 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2018-08-10 16:30:56,143 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2018-08-10 16:30:56,144 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2018-08-10 16:30:56,145 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop'], 'uid': None}
2018-08-10 16:30:56,146 - User['sqoop'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-10 16:30:56,147 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-10 16:30:56,148 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-10 16:30:56,149 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-08-10 16:30:56,151 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2018-08-10 16:30:56,156 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2018-08-10 16:30:56,156 - Group['hdfs'] {}
2018-08-10 16:30:56,156 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop', u'hdfs']}
2018-08-10 16:30:56,157 - FS Type: HDFS
2018-08-10 16:30:56,157 - Directory['/etc/hadoop'] {'mode': 0755}
2018-08-10 16:30:56,158 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2018-08-10 16:30:56,169 - Repository['HDP-3.0-repo-2'] {'append_to_file': False, 'base_url': 'http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-2', 'mirror_list': None}
2018-08-10 16:30:56,175 - File['/etc/yum.repos.d/ambari-hdp-2.repo'] {'content': '[HDP-3.0-repo-2]\nname=HDP-3.0-repo-2\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-08-10 16:30:56,176 - Writing File['/etc/yum.repos.d/ambari-hdp-2.repo'] because contents don't match
2018-08-10 16:30:56,176 - Repository['HDP-3.0-GPL-repo-2'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.0.0.0', 'action': ['create'], 'components': [u'HDP-GPL', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-2', 'mirror_list': None}
2018-08-10 16:30:56,179 - File['/etc/yum.repos.d/ambari-hdp-2.repo'] {'content': '[HDP-3.0-repo-2]\nname=HDP-3.0-repo-2\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-3.0-GPL-repo-2]\nname=HDP-3.0-GPL-repo-2\nbaseurl=http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.0.0.0\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-08-10 16:30:56,179 - Writing File['/etc/yum.repos.d/ambari-hdp-2.repo'] because contents don't match
2018-08-10 16:30:56,180 - Repository['HDP-UTILS-1.1.0.22-repo-2'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-2', 'mirror_list': None}
2018-08-10 16:30:56,182 - File['/etc/yum.repos.d/ambari-hdp-2.repo'] {'content': '[HDP-3.0-repo-2]\nname=HDP-3.0-repo-2\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-3.0-GPL-repo-2]\nname=HDP-3.0-GPL-repo-2\nbaseurl=http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.0.0.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-UTILS-1.1.0.22-repo-2]\nname=HDP-UTILS-1.1.0.22-repo-2\nbaseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-08-10 16:30:56,183 - Writing File['/etc/yum.repos.d/ambari-hdp-2.repo'] because contents don't match
2018-08-10 16:30:56,183 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-08-10 16:30:56,429 - Skipping installation of existing package unzip
2018-08-10 16:30:56,429 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-08-10 16:30:56,529 - Skipping installation of existing package curl
2018-08-10 16:30:56,530 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-08-10 16:30:56,630 - Skipping installation of existing package hdp-select
Command failed after 1 tries
... View more
Labels: