Member since
06-26-2019
68
Posts
8
Kudos Received
6
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3428 | 04-20-2018 09:51 PM | |
| 4238 | 02-08-2018 01:27 AM | |
| 1316 | 01-31-2018 06:39 PM | |
| 5345 | 01-31-2018 07:27 AM | |
| 1380 | 01-17-2018 09:37 PM |
01-17-2018
09:37 PM
looks like somehow a corrupt version got into our repo. deleted it and issues a new request using yum install which got the latest version from the mirror and things were fine again.
... View more
01-17-2018
07:17 PM
in HDP 2.6.1 oozie has dependency on bigtop tomcat but that rpm cannot be downloaded ... ====================================================================================================================================================================================
Package Arch Version Repository Size
====================================================================================================================================================================================
Installing:
oozie_2_6_4_0_91 noarch 4.2.0.2.6.4.0-91 HDP-2.6-repo-1 4.9 k
Installing for dependencies:
bigtop-tomcat noarch 6.0.48-1 HDP-2.6-repo-1 5.5 M
oozie_2_6_4_0_91-client noarch 4.2.0.2.6.4.0-91 HDP-2.6-repo-1 28 M
oozie_2_6_4_0_91-common noarch 4.2.0.2.6.4.0-91 HDP-2.6-repo-1 121 M
results in something like this bigtop-tomcat-6.0.48-1.noarch.rpm | 5.5 MB 00:00
http://myserver:8081/artifactory/hortonworks-hdp/bigtop-tomcat/bigtop-tomcat-6.0.48-1.noarch.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=HDP-2.6-repo-1 clean metadata
Trying other mirror.
Error Downloading Packages:
bigtop-tomcat-6.0.48-1.noarch: failure: bigtop-tomcat/bigtop-tomcat-6.0.48-1.noarch.rpm from HDP-2.6-repo-1: [Errno 256] No more mirrors to try. Is there something wrong with this rpm ? Do I need to import signatures ? $ rpm --checksig bigtop-tomcat-6.0.48-1.noarch.rpm
bigtop-tomcat-6.0.48-1.noarch.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING KEYS: (MD5) PGP#07513cad)
$ rpm --checksig --nogpg bigtop-tomcat-6.0.48-1.noarch.rpm
bigtop-tomcat-6.0.48-1.noarch.rpm: sha1 md5 OK
... View more
Labels:
01-17-2018
06:41 PM
@Aditya Sirna So installing the cluster from the public repositories did work, however when switching back to my local repos it seems that the oozie tomcat is being picked up from the bigtop repository rather than the HDP repository specified in the VDF. My internal repository is set to point to http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.4.0 for hdp components. and a yum info oozie_2_6_4_0_91 shows that it is available from the HDP-2.6-repo-1 repo in pointing to my internal repo... 2018-01-17 18:15:24,397 - The 'oozie-client' component did not advertise a version. This may indicate a problem with the component packaging. However, the stack-select tool was able to report a single version installed (2.6.4.0-91). This is the version that will be reported.
2018-01-17 18:16:28,013 - The 'oozie-client' component did not advertise a version. This may indicate a problem with the component packaging. However, the stack-select tool was able to report a single version installed (2.6.4.0-91). This is the version that will be reported.
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py", line 71, in <module>
OozieClient().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py", line 33, in install
self.install_packages(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 821, in install_packages
retry_count=agent_stack_retry_count)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 53, in action_install
self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 264, in install_package
self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 266, in checked_call_with_retries
return self._call_with_retries(cmd, is_checked=True, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 283, in _call_with_retries
code, out = func(cmd, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install oozie_2_6_4_0_91' returned 1. Error Downloading Packages:
bigtop-tomcat-6.0.48-1.noarch: failure: bigtop-tomcat/bigtop-tomcat-6.0.48-1.noarch.rpm from HDP-2.6-repo-1: [Errno 256] No more mirrors to try.
... View more
01-17-2018
06:35 PM
Well that did work. However as I am doing a blueprint install what this means is that as soon as the metrics collector is installed I would need to run this script. Is there a hook in the python scripts where I can inject this command to be run so that I dont have to wait for the services to fail to start then ansibilize and run this command on all the nodes and then issue a restart command ?
... View more
01-17-2018
06:42 AM
I had tried that but it did not work at the core the issue was somewhat related to this question which does not have an official answer. Instead, I have moved on to ambari 2.6.1 and the HDP 2.6.4 stack. The cluster installed successfully but did not start because the METRICS_MONITOR service could not start and that problem looks similar to this. sigh ! Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_monitoring/main.py", line 27, in <module>
from core.controller import Controller
File "/usr/lib/python2.6/site-packages/resource_monitoring/core/controller.py", line 27, in <module>
from metric_collector import MetricsCollector
File "/usr/lib/python2.6/site-packages/resource_monitoring/core/metric_collector.py", line 23, in <module>
from host_info import HostInfo
File "/usr/lib/python2.6/site-packages/resource_monitoring/core/host_info.py", line 22, in <module>
import psutil
File "/usr/lib/python2.6/site-packages/resource_monitoring/psutil/build/lib.linux-x86_64-2.7/psutil/__init__.py", line 89, in <module>
import psutil._pslinux as _psplatform
File "/usr/lib/python2.6/site-packages/resource_monitoring/psutil/build/lib.linux-x86_64-2.7/psutil/_pslinux.py", line 20, in <module>
from psutil import _common
ImportError: cannot import name _common
psutil binaries need to be built by running, psutil/build.py
... View more
01-16-2018
09:19 PM
@Aditya Sirna Thanks, I assume you mean that I would need to change that for ALL components. In the example above the datanode install required the hadooplzo compression libararies but the version available for both of them would be different. I guess the above jira is private is this fix present in Ambari 2.6.0.0 ? Nevertheless changing the version for each component would be quite tedious to get right even after it is automated to be fixed on ALL the nodes for ALL the components. Is there a better fix for this ? so that the stack_version is picked up as from the registered version ? I am unable to install the cluster using blueprints or the ui with this bug in place .
... View more
01-16-2018
07:06 AM
What is the correct way of configuring a custom repo (say using jfrog) which is mirroring the public repo so that a blueprint install can find packages ?
for instance my /etc/yum.repos.d/ambari-hdp-1.repo is [HDP-2.6-repo-1]
name=HDP-2.6-repo-1
baseurl=http://myserver:8081/artifactory/hortonworks-hdp/
path=/
enabled=1
gpgcheck=0
[HDP-UTILS-1.1.0.21-repo-2]
name=HDP-UTILS-1.1.0.21-repo-2
baseurl=http://myserver:8081/artifactory/hortonworks-hdp-utils/
path=/
enabled=1
gpgcheck=0
So if i run sudo yum info hadoop-hdfs-datanode it lists the right package. As part of the 2.6 change in blueprints it is required to register the stack version following the methodology described here by registering a vdf file Once done, I can see my repository_version with id 1 on /api/v1/stacks/HDP/versions/2.6/repository_versions/1/operating_systems/redhat6/repositories/HDP-2.6
{
"href" : "https://fooserver:8443/api/v1/stacks/HDP/versions/2.6/repository_versions/1/operating_systems/redhat6/repositories/HDP-2.6",
"Repositories" : {
"applicable_services" : [ ],
"base_url" : "http://myserver:8081/artifactory/hortonworks-hdp/",
"components" : null,
"default_base_url" : "",
"distribution" : null,
"latest_base_url" : "",
"mirrors_list" : "",
"os_type" : "redhat6",
"repo_id" : "HDP-2.6",
"repo_name" : "HDP",
"repository_version_id" : 1,
"stack_name" : "HDP",
"stack_version" : "2.6",
"unique" : false
}
}
However after registering the cluster template with "repository_version_id" : 1 the cluster initiation request fails to find the repos on any of the nodes. Where exactly do I specify the version so that the blueprint install can pick it up ?? Have also attempted to set the repo version in /var/lib/ambari-server/resources/stacks/HDP/2.6/repos/repoinfo.xml but to no avail. 2018-01-16 06:35:50,800 - Unable to load available packages
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 771, in load_available_packages
self.available_packages_in_repos = pkg_provider.get_available_packages_in_repos(repos)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 85, in get_available_packages_in_repos
available_packages.extend(self._get_available_packages(repo))
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 146, in _get_available_packages
return self._lookup_packages(cmd, 'Available Packages')
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 191, in _lookup_packages
if items[i + 2].find('@') == 0:
IndexError: list index out of range
2018-01-16 06:35:51,308 - The 'hadoop-hdfs-datanode' component did not advertise a version. This may indicate a problem with the component packaging.
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py", line 155, in <module>
DataNode().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py", line 48, in install
import params
File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/params.py", line 25, in <module>
from params_linux import *
File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py", line 391, in <module>
lzo_packages = get_lzo_packages(stack_version_unformatted)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_lzo_packages.py", line 45, in get_lzo_packages
lzo_packages += [script_instance.format_package_name("hadooplzo_${stack_version}"),
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 538, in format_package_name
raise Fail("Cannot match package for regexp name {0}. Available packages: {1}".format(name, self.available_packages_in_repos))
resource_management.core.exceptions.Fail: Cannot match package for regexp name hadooplzo_${stack_version}. Available packages: []
2018-01-16 06:36:04,670 - Unable to load available packages
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 771, in load_available_packages
self.available_packages_in_repos = pkg_provider.get_available_packages_in_repos(repos)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 85, in get_available_packages_in_repos
available_packages.extend(self._get_available_packages(repo))
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 146, in _get_available_packages
return self._lookup_packages(cmd, 'Available Packages')
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 191, in _lookup_packages
if items[i + 2].find('@') == 0:
IndexError: list index out of range
2018-01-16 06:36:05,184 - The 'hadoop-hdfs-datanode' component did not advertise a version. This may indicate a problem with the component packaging.
... View more
Labels:
- Labels:
-
Apache Ambari
01-02-2018
08:58 PM
@Jay Kumar SenSharma it seems that my console had kind of eaten up the "-" from the "-d@" curl command. ... please disregard .. will update once figured out .. This does not work although it did seem to. Starting from a clean environment leads me to the same state. i.e
Register the VDF (local) Register the Blueprint with the stack_name and stack_version Initiate a cluster deployment using a cluster template using the repository_version_id or repository_version Still yields the same error "message" : "Stack information should be provided when creating a cluster" My previous work-around was to restart the ambari-agent nodes on each node and that seemed to work. But that doesnt seem to work as of today. From the ambar-server log 02 Jan 2018 20:57:17,595 ERROR [ambari-client-thread-85] BaseManagementHandler:67 - Bad request received: Stack information should be provided when creating a cluster<br>
... View more
12-21-2017
10:36 PM
Hi After installation of a cluster using the Ambari UI i exported the blueprint. Then attempted to re-install the same cluster using the blueprint however i receive this error when apply the cluster template. "status" : 400,
"message" : "Stack information should be provided when creating a cluster" To be sure the blueprint contains the following information "Blueprints" : {
"blueprint_name" : "cluster-blueprint",
"stack_name" : "HDP",
"stack_version" : "2.6"
}
How do we provide this information ? I see that this commit causes this . What is the workaround ?
... View more
Labels:
- Labels:
-
Apache Ambari
12-19-2017
07:36 AM
1 Kudo
so this works; probably the documentation needs to be updated curl -vX POST -u admin:admin --header "X-Requested-By: ambari"-d '{"ServiceInfo":{"service_name":"ZOOKEEPER", "desired_repository_version_id":1}}' http://myhost:8080/api/v1/clusters/testCluster/services
... View more
- « Previous
- Next »