installing ambari with local repository and no internet access

[#1]

I am following Hortonworks document for this and they are mentioning and have setup the ambari.repo file but install is failing , please see what I am doing wrong, also attaching the screenshot showing that the repository directory is accessable from web.

getting following error:

http://hadoop1/AMBARI-2.4.2.0/centos6/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 504 Gateway Timeout"
Trying other mirror.

the repond.xml file is there and accessable

[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]# ls /var/www/html/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml
/var/www/html/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml
[root@hadoop1 yum.repos.d]#

though the document is saying to set the base url as "http://<web.server>/Ambari-2.4.2.0/<OS> " but the respomd.xml file is the <OS><RELEASE> directory , I tried both but no luck.

the ambari.repo file is shown below:

[root@hadoop1 yum.repos.d]# more ambari.repo
#VERSION_NUMBER=2.4.2.0-22
[Updates-ambari-2screenshot.jpg.4.2.0]
name=ambari-2.4.2.0 - Updates
baseurl=http://hadoop1/AMBARI-2.4.2.0/centos6/2.4.2.0-136
gpgcheck=0
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[#2]

The yum repolist works for other repositories but not the local ambari .

[root@hadoop1 yum.repos.d]# yum repolist
Loaded plugins: fastestmirror, priorities, protectbase, refresh-packagekit, security
Loading mirror speeds from cached hostfile
epel/metalink                                                                                            |  13 kB     00:00
 * base: centos.unixheads.org
 * epel: archive.linux.duke.edu
 * extras: centos.mirror.constant.com
 * updates: centos.mirror.lstn.net
http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 504 Gateway Timeout"
Trying other mirror.
http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 504 Gateway Timeout"
Trying other mirror.
base                                                                                                     | 3.7 kB     00:00
base/primary_db                                                                                          | 4.7 MB     00:00
centos-sclo-rh                                                                                           | 2.9 kB     00:00
centos-sclo-rh/primary_db                                                                                | 2.5 MB     00:00
epel                                                                                                     | 4.3 kB     00:00
epel/primary_db                                                                                          | 5.9 MB     00:00
extras                                                                                                   | 3.4 kB     00:00
extras/primary_db                                                                                        |  29 kB     00:00
updates                                                                                                  | 3.4 kB     00:00
updates/primary_db                                                                                       | 1.4 MB     00:00
repo id                                          repo name                                                                status
HDP-2.5                                          HDP-2.5                                                                     200
HDP-2.5.0.0                                      HDP-2.5.0.0                                                                 200
HDP-UTILS-1.1.0.21                               HDP-UTILS-1.1.0.21                                                           56
HDP-UTILS-2.5.0.0                                HDP-UTILS-2.5.0.0                                                            56
Updates-ambari-2.4.2.0                           ambari-2.4.2.0 - Updates                                                      0
base                                             CentOS-6 - Base                                                           6,706
centos-sclo-rh                                   CentOS-6 - SCLo rh                                                        5,553
epel                                             Extra Packages for Enterprise Linux 6 - x86_64                           12,309
extras                                           CentOS-6 - Extras                                                            45
updates                                          CentOS-6 - Updates                                                          354
repolist: 25,479
[root@hadoop1 yum.repos.d]#

[#3 · Accepted solution]

@Sami Ahmad

Looks like you have another repo file present inside your "/etc/yum.repos.d" directory which has the following Host entry in it:

http://hadoop1.tolls.dot.state.fl.us

.

Can you please try to grep that host name from where is it coming from?

# cd /etc/yum.repos.d
# grep 'hadoop1.tolls.dot.state.fl.us' *.repo

(OR)

# find . -name "*.repo" | xargs grep -i "hadoop1.tolls.dot.state.fl.us"

.

In the output of the above command if you find any such hostname then move that repo file somewhere else. Because that seems to be pointing to an Incorrect Ambari repo.

[#4]

I only have this host name in one file ambari.repo and as I am showing above the url is reachable

[root@hadoop1 yum.repos.d]# grep 'hadoop1.tolls.dot.state.fl.us' *.repo
ambari.repo:baseurl=http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136
[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]# more ambari.repo
#VERSION_NUMBER=2.4.2.0-22
[ambari-2.4.2.0]
name=ambari-2.4.2.0
baseurl=http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136
gpgcheck=0
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@hadoop1 yum.repos.d]#

[#5]

@Sami Ahmad

Are you accessing the mentioned URL using browser? Do you have any proxy_url (proxy host/port) configured on your browser?

If yes, then you should make sure that from the host where you have that repo it is configured to be accessible via the same proxy.

For example if you do a curl command from the host where you have that repo then what happens?

# wget http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml
OR

# curl http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xm

.

If it fails then you should check if you have any proxy configured in the file "/etc/yum.conf"

Example:

# cat /etc/yum.conf  | grep proxy
proxy=http://yourproxy.test.com:8080

.

[#6]

hi Jay

please see below , proxy is set but wget times out

also see the attached screenshot showing that from web the URL is reachable screenshot.jpg

[root@hadoop1 yum.repos.d]# wget http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml
--2017-06-12 13:22:57--  http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml
Resolving dotatofwproxy.tolls.dot.state.fl.us... 10.100.30.27
Connecting to dotatofwproxy.tolls.dot.state.fl.us|10.100.30.27|:8080... connected.
Proxy request sent, awaiting response... 504 Gateway Timeout
2017-06-12 13:23:06 ERROR 504: Gateway Timeout.
[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]# cat /etc/yum.conf | grep proxy
proxy=http://dotatofwproxy.tolls.dot.state.fl.us:8080
[root@hadoop1 yum.repos.d]#

[#7]

@Sami Ahmad

Are you sure that your Proxy Host and Port is the same that you are using in your "/etc/yum.conf" ? (I just gave an example, but in your case the proxy host & port might be different). You need to check first if your browser is able to access that file or not and if yes then check if your browser is using any proxy?

Following is just example host & port.

# cat /etc/yum.conf  | grep proxy
proxy=http://yourproxy.test.com:808

[#8]

also if it was a proxy issue how can I access the public url and not the private one ?

in output below , the first repo file (ambari.repo.bk) works .

the second file (ambari.repo) does not .

[root@hadoop1 yum.repos.d]# more ambari.repo.bk
#VERSION_NUMBER=2.4.1.0-22
[Updates-ambari-2.4.1.0]
name=ambari-2.4.1.0 - Updates
baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.1.0
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]#
[root@hadoop1 yum.repos.d]# more ambari.repo
#VERSION_NUMBER=2.4.2.0-22
[ambari-2.4.2.0]
name=ambari-2.4.2.0
baseurl=http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136
gpgcheck=0
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@hadoop1 yum.repos.d]#

[#9]

@Sami Ahmad

You are getting Http Response 504

http://hadoop1.tolls.dot.state.fl.us/AMBARI-2.4.2.0/centos6/2.4.2.0-136/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 504 Gateway Timeout"

According to Http RFC specification : https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.5.5

10.5.5 504 Gateway Timeout
-------------------------------

   The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI (e.g. HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed to access in attempting to complete the request.

   Note: Note to implementors: some deployed proxies are known to return 400 or 500 when DNS lookups time out.

.

[#10]

how is this url passing then through proxy ?

http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.1.0