Member since
09-28-2017
88
Posts
3
Kudos Received
0
Solutions
07-28-2022
05:00 AM
apt-get update Hit:1 https://repo.pbis.beyondtrust.com/apt pbiso InRelease Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] Hit:4 http://apt.postgresql.org/pub/repos/apt bionic-pgdg InRelease Hit:5 http://packages.microsoft.com/repos/code stable InRelease Get:6 http://archive.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] Hit:7 https://packages.microsoft.com/repos/vscode stable InRelease Err:8 http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18 HDP-UTILS InRelease 403 Forbidden [IP: 65.9.112.104 80] Err:9 http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0 HDP InRelease 403 Forbidden [IP: 65.9.112.104 80] Get:10 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,339 kB] Err:11 http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0 HDP-GPL InRelease 403 Forbidden [IP: 65.9.112.104 80] Get:12 http://archive.ubuntu.com/ubuntu bionic-security/main Translation-en [407 kB] Get:13 http://archive.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [832 kB] Get:14 http://archive.ubuntu.com/ubuntu bionic-security/restricted Translation-en [114 kB] Reading package lists... Done N: See apt-secure(8) manpage for repository creation and user configuration details. N: Updating from such a repository can't be done securely, and is therefore disabled by default. E: The repository 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18 HDP-UTILS InRelease' is not signed. E: Failed to fetch http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18/dists/HDP-UTILS/InRelease 403 Forbidden [IP: 65.9.112.104 80] E: Failed to fetch http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/dists/HDP/InRelease 403 Forbidden [IP: 65.9.112.104 80] E: The repository 'http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0 HDP InRelease' is not signed. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details. E: Failed to fetch http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0/dists/HDP-GPL/InRelease 403 Forbidden [IP: 65.9.112.104 80] E: The repository 'http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0 HDP-GPL InRelease' is not signed. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details.
... View more
07-28-2022
01:22 AM
I have exactly the same files installed on all hosts . Usually when we buy new server we clone the entire host, and initiate install from ambari ui, because everything is already installed all it does in reconfigure files. but the repository is no longer accessible from any host
... View more
07-27-2022
02:25 PM
HDP-3.1.0.0 2022-07-28 00:19:45,289 - Package Manager failed to install packages: No package found for hive-${stack_version}(expected name: hive-3-1-0-0-78)
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 486, in <module>
InstallPackages().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 149, in actionexecute
raise Fail("Failed to distribute repositories/install packages")
resource_management.core.exceptions.Fail: Failed to distribute repositories/install packages
stdout: /var/lib/ambari-agent/data/output-6991.txt
2022-07-28 00:18:33,442 - Will install packages for repository version 3.1.0.0-78
2022-07-28 00:18:33,443 - Repository['HDP-UTILS-1.1.0.22-repo-1'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18', 'action': ['prepare'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2022-07-28 00:18:33,450 - Repository['HDP-3.1-repo-1'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0', 'action': ['prepare'], 'components': [u'HDP', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2022-07-28 00:18:33,452 - Repository['HDP-3.1-GPL-repo-1'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0', 'action': ['prepare'], 'components': [u'HDP-GPL', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2022-07-28 00:18:33,454 - Repository[None] {'action': ['create']}
2022-07-28 00:18:33,456 - File['/tmp/tmprKy1g1'] {'content': 'deb http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18 HDP-UTILS main\ndeb http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0 HDP main\ndeb http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0 HDP-GPL main'}
2022-07-28 00:18:33,883 - Writing File['/tmp/tmprKy1g1'] because contents don't match
2022-07-28 00:18:33,988 - File['/tmp/tmp5bLc_I'] {'content': StaticFile('/etc/apt/sources.list.d/ambari-hdp-1.list')}
2022-07-28 00:18:34,650 - Writing File['/tmp/tmp5bLc_I'] because contents don't match
2022-07-28 00:18:34,767 - call[('ambari-python-wrap', u'/usr/bin/hdp-select', 'versions')] {}
2022-07-28 00:18:34,866 - call returned (0, '3.1.0.0-78')
2022-07-28 00:18:34,868 - Temporary sources directory was created: /tmp/tmpJmw3Rs-ambari-apt-sources-d
2022-07-28 00:18:34,868 - Temporary sources file will be copied: /tmp/tmpJmw3Rs-ambari-apt-sources-d/ambari-hdp-1.list
2022-07-28 00:18:35,043 - Installing package hdp-select ('/usr/bin/apt-get -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install -o Dir::Etc::SourceList=/dev/null -o Dir::Etc::SourceParts= hdp-select')
2022-07-28 00:18:35,416 - Removing temporary sources file: /tmp/tmpJmw3Rs-ambari-apt-sources-d/ambari-hdp-1.list
2022-07-28 00:18:35,417 - Removing temporary sources directory: /tmp/tmpJmw3Rs-ambari-apt-sources-d
2022-07-28 00:19:22,785 - Looking for matching packages in the following repositories: public-repo-1.hortonworks.com_HDP-UTILS-1.1.0.22_repos_ubuntu18, public-repo-1.hortonworks.com_HDP_ubuntu18_3.x_updates_3.1.0.0, public-repo-1.hortonworks.com_HDP-GPL_ubuntu18_3.x_updates_3.1.0.0
2022-07-28 00:19:22,788 - Command repositories: HDP-UTILS-1.1.0.22-repo-1, HDP-3.1-repo-1, HDP-3.1-GPL-repo-1
2022-07-28 00:19:22,788 - Applicable repositories: HDP-UTILS-1.1.0.22-repo-1, HDP-3.1-repo-1, HDP-3.1-GPL-repo-1
2022-07-28 00:19:45,280 - Looking for matching packages in the following repositories: public-repo-1.hortonworks.com_HDP-UTILS-1.1.0.22_repos_ubuntu18, public-repo-1.hortonworks.com_HDP_ubuntu18_3.x_updates_3.1.0.0, public-repo-1.hortonworks.com_HDP-GPL_ubuntu18_3.x_updates_3.1.0.0
2022-07-28 00:19:45,289 - Package Manager failed to install packages: No package found for hive-${stack_version}(expected name: hive-3-1-0-0-78)
2022-07-28 00:20:10,532 - Installation of packages failed. Checking if installation was partially complete
2022-07-28 00:20:10,533 - Old versions: ['3.1.0.0-78']
2022-07-28 00:20:10,534 - call[('ambari-python-wrap', u'/usr/bin/hdp-select', 'versions')] {}
2022-07-28 00:20:10,636 - call returned (0, '3.1.0.0-78')
2022-07-28 00:20:10,637 - New versions: ['3.1.0.0-78']
2022-07-28 00:20:10,637 - Deltas: set([])
Command failed after 1 tries i have an old stable version of amabri with 16 hosts, mostly used for spark over yarn. i want to add few more hosts to the cluster but i cannot install the packages:
... View more
Labels:
- Labels:
-
Apache YARN
-
HDFS
-
Manual Installation
02-20-2019
01:26 PM
i think that did what i needed. thanks
... View more
02-20-2019
12:15 PM
i want to disable second attempt
... View more
02-20-2019
12:03 PM
when submitting a job that failed, it try to run with another attemped, how i can disable the second run ? what configuration param i must set?
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
02-20-2019
11:15 AM
ok, just stopped 3 hive services, still it abit bother me that there are red dots in the ui :), Btw why there are a dependency on hive?
... View more
02-20-2019
11:02 AM
Thanks for the quick response, Can i delete all hive? Hive Metastore HiveServer2 Hive Client Metrics are useful. i can see some bottlenecks and improve the performance, Currently still struggling the improve the time it take to allocate the containers each job ( all the time they are the same and it take 30-60 seconds every job to allocate them ) Btw why it is a requirements (in the install stage it was selected cause i needed spark\yarn)
... View more
02-20-2019
10:37 AM
i have a large cluster installed currently it is used only for spark on yarn jobs what services on the master i can remove\uninstall to free up some ram? one server to accept spark jobs currently have the following services: Timeline Service V1.5 / YARN Master History Server / MapReduce2 Master Hive Metastore / Hive Master HiveServer2 / Hive Master Metrics Collector / Ambari Metrics Master Grafana / Ambari Metrics Master NameNode / HDFS Master ResourceManager / YARN Master SNameNode / HDFS Master Spark2 History Server / Spark2 Master Timeline Service V2.0 Reader / YARN Master YARN Registry DNS / YARN Master ZooKeeper Server / ZooKeeper Master DataNode / HDFS Slave Metrics Monitor / Ambari Metrics Slave HDFS Client / HDFS Client Hive Client / Hive Client MapReduce2 Client / MapReduce2 Client Spark2 Client / Spark2 Client Tez Client / Tez Client YARN Client / YARN Client ZooKeeper Client / ZooKeeper Client and multiple clients with: Metrics Monitor / Ambari Metrics Slave NodeManager / YARN Slave
... View more
Labels:
02-18-2019
11:27 AM
works thanks 🙂
... View more