Ambari Blueprint Installation using local repositories

[#1]

I have set a local repository which is being propagated in the /etc/yum.repos.d directory (centos7/redhat7) on new installed machines. When I set up a new ambari-server and I install a cluster using blueprints, official repositories are being configured in the /etc/yum.repos.d and the local ones that I have set up are skipped.

I tried to modify the file /var/lib/ambari-server/resources/stacks/HDP/2.4/repos/repoinfo.xml in the ambari-server by replacing the redhat7 repository and I have changed the baseurl with the endpoint of the local repository, but installation is still ignoring this endpoint and keeps setting the official baseurl (which I want to avoid).

How can I force the use of local repositories when cluster installation+configuration is performed through blueprints?

Thanks a lot,

Marc

[#2 · Accepted solution]

@Marc Caubet

Call the API to see what configuration is coming back on the repositories:

curl -u username:password http://ambariserver:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories

Is it returning the configuration you modified in repoinfo.xml?

Did you restart Ambari Server after making those changes?

According to the docs you can specify a repo like this:

PUT /api/v1/stacks/:stack/versions/:stackVersion/operating_systems/:osType/repositories/:repoId
 
{
  "Repositories” : {
    "base_url” : ”<CUSTOM_REPO_BASE_URL>",
    "verify_base_url” : true
  }
}

Documentation here: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-Step4:SetupStackRepositorie...

[#3]

Hi Michael thanks for the reply.

[root@ambarisrv yum.repos.d]# curl -u username:password http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories
{
  "href" : "http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories",
  "items" : [
  {
  "href" : "http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories/HDP-2.4",
  "Repositories" : {
  "os_type" : "redhat7",
  "repo_id" : "HDP-2.4",
  "stack_name" : "HDP",
  "stack_version" : "2.4"
  }
  },
  {
  "href" : "http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories/HDP-UTILS-1.1.0.20",
  "Repositories" : {
  "os_type" : "redhat7",
  "repo_id" : "HDP-UTILS-1.1.0.20",
  "stack_name" : "HDP",
  "stack_version" : "2.4"
  }
  }
  ]
}

So this is not returning the info from the repoinfo.xml. I also tried by restarting the ambari-server.

[root@ambarisrv yum.repos.d]# curl -i -H "X-Requested-By: ambari" -u username:password -X PUT -d'{ "Repositories" : { "base_url":"http://antchamber01/hdp", "verify_base_url":true }}' http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories/HDP-2.4
HTTP/1.1 200 OK
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
User: username
Set-Cookie: AMBARISESSIONID=qztzzlsjnykfrct0hcp1jabv;Path=/;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Content-Length: 0
Server: Jetty(8.1.17.v20150415)

This neither helps 😞

[root@ambarisrv yum.repos.d]# curl -u username:password http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories
{
  "href" : "http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories",
  "items" : [
  {
  "href" : "http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories/HDP-2.4",
  "Repositories" : {
  "os_type" : "redhat7",
  "repo_id" : "HDP-2.4",
  "stack_name" : "HDP",
  "stack_version" : "2.4"
  }
  },
  {
  "href" : "http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories/HDP-UTILS-1.1.0.20",
  "Repositories" : {
  "os_type" : "redhat7",
  "repo_id" : "HDP-UTILS-1.1.0.20",
  "stack_name" : "HDP",
  "stack_version" : "2.4"
  }
  }
  ]
}

Thanks!

Marc

[#4]

Hi again,

tying a fresh installation, finally it worked with the workaround pointed by Michael.

curl -i -H "X-Requested-By: ambari" -u username:password -X PUT -d'{ "Repositories" : { "base_url":"http://antchamber01/hdp", "verify_base_url":true }}' http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories/HDP-2.4

curl -i -H "X-Requested-By: ambari" -u username:password -X PUT -d'{ 
"Repositories" : { "base_url":"http://antchamber01/hdp-utils", 
"verify_base_url":true }}' 
http://ambarisrv:8080/api/v1/stacks/HDP/versions/2.4/operating_systems/redhat7/repositories/HDP-UTIL...

Thanks a lot for your help!

Marc

[#5]

@Marc Caubet

Glad I could help!

[#6]

I used Ambari2.6.1 and I tried all of the solution,however,it did not work for me.The repository is still the latest that offical provided.

[#7]

@ke chen
Starting from Ambari 2.6, the way of specifying repo versions has changed as you can see here. You have to register a VDF file like this one with the version_definitions API (in the VDF file, replace the repository-info.os.repo.baseurl by your local repo url).

curl -u user:pwd -H "X-Requested-By: ambari" -X POST
http://ambari.server:8080/api/v1/version_definitions \
-d '{
 "VersionDefinition": {
   "version_url": "your.vdf.file.xml"
 }
}'

You will get an id for the version and you will have to reference it when creating the cluster :

{
  "blueprint": "<blueprint name>",
  "repository_version_id": 1,
  "default_password": "password",
  "host_groups": [ ... ]
}

More details in the release note: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-release-notes/content/ambari_reln...

[#8]

This didn't work at all even at 3.1.  So what I did after my manual hdp cluster, I gather all the binaries with "rpm -qa --last|more", grab all that binaries in a file (excluding the date) ambari.yum.  Before the blueprint I run this "for i in `cat ambari.yum`; do yum install -y $i; done".

It worksssssssssssssssssssssssssssss  but of course not dynamic one.

[#9]

Here's the github with blueprint work around.  There's no ldap/keytab settings yet but reset ambari is included or snapshot it from ur virtualbox or worstation or even esx or script from ur azure images.  

https://github.com/rv-ansible/hadoop_server.git

works with centos/redhat 7.6