Member since
09-28-2017
88
Posts
3
Kudos Received
0
Solutions
07-28-2022
05:00 AM
apt-get update Hit:1 https://repo.pbis.beyondtrust.com/apt pbiso InRelease Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] Hit:4 http://apt.postgresql.org/pub/repos/apt bionic-pgdg InRelease Hit:5 http://packages.microsoft.com/repos/code stable InRelease Get:6 http://archive.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] Hit:7 https://packages.microsoft.com/repos/vscode stable InRelease Err:8 http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18 HDP-UTILS InRelease 403 Forbidden [IP: 65.9.112.104 80] Err:9 http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0 HDP InRelease 403 Forbidden [IP: 65.9.112.104 80] Get:10 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,339 kB] Err:11 http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0 HDP-GPL InRelease 403 Forbidden [IP: 65.9.112.104 80] Get:12 http://archive.ubuntu.com/ubuntu bionic-security/main Translation-en [407 kB] Get:13 http://archive.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [832 kB] Get:14 http://archive.ubuntu.com/ubuntu bionic-security/restricted Translation-en [114 kB] Reading package lists... Done N: See apt-secure(8) manpage for repository creation and user configuration details. N: Updating from such a repository can't be done securely, and is therefore disabled by default. E: The repository 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18 HDP-UTILS InRelease' is not signed. E: Failed to fetch http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18/dists/HDP-UTILS/InRelease 403 Forbidden [IP: 65.9.112.104 80] E: Failed to fetch http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/dists/HDP/InRelease 403 Forbidden [IP: 65.9.112.104 80] E: The repository 'http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0 HDP InRelease' is not signed. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details. E: Failed to fetch http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0/dists/HDP-GPL/InRelease 403 Forbidden [IP: 65.9.112.104 80] E: The repository 'http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0 HDP-GPL InRelease' is not signed. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details.
... View more
07-28-2022
01:22 AM
I have exactly the same files installed on all hosts . Usually when we buy new server we clone the entire host, and initiate install from ambari ui, because everything is already installed all it does in reconfigure files. but the repository is no longer accessible from any host
... View more
07-27-2022
02:25 PM
HDP-3.1.0.0 2022-07-28 00:19:45,289 - Package Manager failed to install packages: No package found for hive-${stack_version}(expected name: hive-3-1-0-0-78)
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 486, in <module>
InstallPackages().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 149, in actionexecute
raise Fail("Failed to distribute repositories/install packages")
resource_management.core.exceptions.Fail: Failed to distribute repositories/install packages
stdout: /var/lib/ambari-agent/data/output-6991.txt
2022-07-28 00:18:33,442 - Will install packages for repository version 3.1.0.0-78
2022-07-28 00:18:33,443 - Repository['HDP-UTILS-1.1.0.22-repo-1'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18', 'action': ['prepare'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2022-07-28 00:18:33,450 - Repository['HDP-3.1-repo-1'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0', 'action': ['prepare'], 'components': [u'HDP', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2022-07-28 00:18:33,452 - Repository['HDP-3.1-GPL-repo-1'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0', 'action': ['prepare'], 'components': [u'HDP-GPL', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2022-07-28 00:18:33,454 - Repository[None] {'action': ['create']}
2022-07-28 00:18:33,456 - File['/tmp/tmprKy1g1'] {'content': 'deb http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18 HDP-UTILS main\ndeb http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0 HDP main\ndeb http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0 HDP-GPL main'}
2022-07-28 00:18:33,883 - Writing File['/tmp/tmprKy1g1'] because contents don't match
2022-07-28 00:18:33,988 - File['/tmp/tmp5bLc_I'] {'content': StaticFile('/etc/apt/sources.list.d/ambari-hdp-1.list')}
2022-07-28 00:18:34,650 - Writing File['/tmp/tmp5bLc_I'] because contents don't match
2022-07-28 00:18:34,767 - call[('ambari-python-wrap', u'/usr/bin/hdp-select', 'versions')] {}
2022-07-28 00:18:34,866 - call returned (0, '3.1.0.0-78')
2022-07-28 00:18:34,868 - Temporary sources directory was created: /tmp/tmpJmw3Rs-ambari-apt-sources-d
2022-07-28 00:18:34,868 - Temporary sources file will be copied: /tmp/tmpJmw3Rs-ambari-apt-sources-d/ambari-hdp-1.list
2022-07-28 00:18:35,043 - Installing package hdp-select ('/usr/bin/apt-get -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install -o Dir::Etc::SourceList=/dev/null -o Dir::Etc::SourceParts= hdp-select')
2022-07-28 00:18:35,416 - Removing temporary sources file: /tmp/tmpJmw3Rs-ambari-apt-sources-d/ambari-hdp-1.list
2022-07-28 00:18:35,417 - Removing temporary sources directory: /tmp/tmpJmw3Rs-ambari-apt-sources-d
2022-07-28 00:19:22,785 - Looking for matching packages in the following repositories: public-repo-1.hortonworks.com_HDP-UTILS-1.1.0.22_repos_ubuntu18, public-repo-1.hortonworks.com_HDP_ubuntu18_3.x_updates_3.1.0.0, public-repo-1.hortonworks.com_HDP-GPL_ubuntu18_3.x_updates_3.1.0.0
2022-07-28 00:19:22,788 - Command repositories: HDP-UTILS-1.1.0.22-repo-1, HDP-3.1-repo-1, HDP-3.1-GPL-repo-1
2022-07-28 00:19:22,788 - Applicable repositories: HDP-UTILS-1.1.0.22-repo-1, HDP-3.1-repo-1, HDP-3.1-GPL-repo-1
2022-07-28 00:19:45,280 - Looking for matching packages in the following repositories: public-repo-1.hortonworks.com_HDP-UTILS-1.1.0.22_repos_ubuntu18, public-repo-1.hortonworks.com_HDP_ubuntu18_3.x_updates_3.1.0.0, public-repo-1.hortonworks.com_HDP-GPL_ubuntu18_3.x_updates_3.1.0.0
2022-07-28 00:19:45,289 - Package Manager failed to install packages: No package found for hive-${stack_version}(expected name: hive-3-1-0-0-78)
2022-07-28 00:20:10,532 - Installation of packages failed. Checking if installation was partially complete
2022-07-28 00:20:10,533 - Old versions: ['3.1.0.0-78']
2022-07-28 00:20:10,534 - call[('ambari-python-wrap', u'/usr/bin/hdp-select', 'versions')] {}
2022-07-28 00:20:10,636 - call returned (0, '3.1.0.0-78')
2022-07-28 00:20:10,637 - New versions: ['3.1.0.0-78']
2022-07-28 00:20:10,637 - Deltas: set([])
Command failed after 1 tries i have an old stable version of amabri with 16 hosts, mostly used for spark over yarn. i want to add few more hosts to the cluster but i cannot install the packages:
... View more
Labels:
- Labels:
-
Apache YARN
-
HDFS
-
Manual Installation
02-25-2021
12:18 AM
bumping again, still trying to disable all application core dump that run inside the containers
... View more
01-21-2021
12:46 AM
we are using ambari, but it is not what i wanted. our hosts are ubuntu 18.04. i tried the following: set ulimit -c 0, set disable core dump via code for our application: #include <sys/types.h>
#include <sys/time.h>
#include <sys/resource.h>
void spc_limit_core(void) {
struct rlimit rlim;
rlim.rlim_cur = rlim.rlim_max = 0;
setrlimit(RLIMIT_CORE, &rlim);
} but when there is an error,( out of memory) an assert is generated and then a coredump,
... View more
01-20-2021
04:46 AM
Where exactly? i am using versions:: HDFS 3.1.1 YARN 3.1.1 MapReduce2 3.1.1 Tez 0.9.1 Hive 3.1.0 HBase 2.0.2 Pig 0.16.0 Sqoop 1.4.7 Oozie 4.3.1 ZooKeeper 3.4.6 Storm 1.2.1 Accumulo 1.7.0 Infra Solr 0.1.0 Ambari Metrics 0.1.0 Atlas 1.1.0 Kafka 2.0.0 Knox 1.0.0 Log Search 0.5.0 Ranger 1.2.0 Ranger KMS 1.2.0 SmartSense 1.5.1.2.7.3.0-139 Spark2 2.3.2 Zeppelin Notebook 0.8.0 Druid 0.12.1 Kerberos 1.10.3-30 Superset 0.23.0
... View more
01-20-2021
01:41 AM
how i can disable all cores generated by applications? we are based on spark\yarn
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
08-18-2020
04:17 AM
i am trying to get the following flow submit a grid app (with about 300 jobs ) sometimes jobs fails and we would like to retry them but after 20 retries (or total 20 failures) we would like to mark the entire application as failed and stop all running\pending jobs i have set yarn.resourcemanager.am.max-attempts=2 how i can limit the total error before marking the entire job submit as failure? i am based spark\yarn
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Spark
-
Apache YARN