Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Passing Proxy from Ambari to HDP.repo and HDP-UTILS.repo

Passing Proxy from Ambari to HDP.repo and HDP-UTILS.repo

Contributor

I'm installing HDP 2.5 using Ambari 2.4.1.0 in RHEL 6.8. My Azure node has limited connectivity and I was permitted by the infrastructure guys to use a proxy only for repository purposes. They didn't allow me to universally set the proxy inside /etc/yum.conf, but only on specific repo files.

In the steps provided here,

https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.1.0/bk_ambari-installation/content/download_the...

I added a proxy variable at the end of ambari.repo file

proxy=http://<proxyhost>:80/

Which allowed me to install Ambari Server and agent.

In addition, to enable the Ambari Server Launch Wizard to detect the HDP stack repo, I followed the instruction here to set the proxy:

http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.1.0/bk_ambari-reference/content/ch_setting_up_an...

which says, I need to add

-Dhttp.proxyHost=<proxyhost> -Dhttp.proxyPort=80

inside the AMBARI_JVM_ARGS inside of /var/lib/ambari-server/ambari-env.sh file, which worked.

However, once I get to the part where each HDP services / components are actually being installed

https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.1.0/bk_ambari-installation/content/install_star...

it looks like the proxy is not detected. This step automatically generates a HDP.repo and a HDP-UTILS.repo files inside /etc/yum.repos.d, which do not inherit the proxy set in ambari.repo or /var/lib/ambari-server/ambari-env.sh. I tried rummaging through /var/lib/ambari-server/resources/stacks/HDP/2.5/repos/repoinfo.xml file, but it seems there is no place I can write the proxy as a hack unlike here on older version:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.0/bk_ambari_reference_guide/content/_configur...

Is there anyway I can insert or plugin the proxy in auto-generated HDP.repo and HDP-UTILS.repo files?

5 REPLIES 5

Re: Passing Proxy from Ambari to HDP.repo and HDP-UTILS.repo

If needed the in the "~/.bash_profile" profile we can add the following:

export http_proxy=http://proxyhosr:3128/ 
export https_proxy=${http_proxy} 
export ftp_proxy=${http_proxy} 

OR

you may want to add the proxy information in the file "/etc/yum.conf " as following:

# cat /etc/yum.conf 
[main] 
cachedir=/var/cache/yum/$basearch/$releasever 
keepcache=0 
debuglevel=2 
logfile=/var/log/yum.log 
exactarch=1 
obsoletes=1 
gpgcheck=1 
plugins=1 
installonly_limit=3 
proxy=http://yourproxy.test.com:8080


Re: Passing Proxy from Ambari to HDP.repo and HDP-UTILS.repo

Super Collaborator

You're correct that Ambari will generate and keep on generating the HDP.repo file, even if you edit it. However, Ambari uses a template to generate the repo files; it's stored in cluster-env/repo_suse_rhel_template

You'll want to update this configuration property to include your proxy=http://<proxyhost>:80/ entry. Then, Amabri will create the repo correctly.

Re: Passing Proxy from Ambari to HDP.repo and HDP-UTILS.repo

Contributor

This is for 2.0.6 version, but not for HDP 2.5

Re: Passing Proxy from Ambari to HDP.repo and HDP-UTILS.repo

Super Collaborator

I'm not sure I follow your comment. You said that the problem you're having affects Ambari 2.4.1 on HDP 2.5 - I'm not sure what HDP 2.0.6 has anything to do with this.

The property I referenced is the template that Ambari uses to write out the HDP.repo file. If you change it to add your proxy, then Ambari will write out the proxy too.

Re: Passing Proxy from Ambari to HDP.repo and HDP-UTILS.repo

@J. D. Bacolod

Try to disable the proxy setting for HDP repo using the command provided

# /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -s get <ambari-server hostname> <clustername> cluster-env >cluster-env.bkp 
# /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -s set <ambari-server hostname> <clustername> cluster-env repo_suse_rhel_template "[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0\nproxy=_none_" 

Once this done, you may do a "yum clean all" on all the hosts

You confirm the change has taken place, the HDP repo should have the line containing "proxy=_none_" in it after trying to install.