Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP Stack Repositories not found

HDP Stack Repositories not found

Expert Contributor

stack-repos.pngHello

I am installing a new HDP cluster. I am using the default stack repositories links provided by ambari, however i am getting a connection not found error (check attached image).

I have tried the stack repositories provided in the following link but none of them worked.

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_Installing_HDP_AMB/content/_hdp_24_repos...

Please advise

Regards

15 REPLIES 15

Re: HDP Stack Repositories not found

@Ahmad Debbas

I just tried HDP 2.4.0.0 repro and its working fine, can you please check if you have internet connectivity to access those repo from Ambari node or any firewall blocking it?

Re: HDP Stack Repositories not found

Expert Contributor

would you please provide me with the link you have tried? thanks! @Jitendra Yadav

Highlighted

Re: HDP Stack Repositories not found

Guru

Can you also confirm if your hosts have access to the Internet? FWIW: If you do not have access to the Internet, you will need setup Local Repository:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_Installing_HDP_AMB/content/_setting_up_a...

Re: HDP Stack Repositories not found

Expert Contributor

I do have access to the internet. I downloaded Ambari 2.2, set it up and now i am installing an HDP cluster. As seen in the image attached, none of the repositories has been accepted. stack-repos.png

Re: HDP Stack Repositories not found

Expert Contributor

It seems there is something between your host and the HW repository blocking the internet connection. It could be your corporate firewall. Can you run following command from one of the hosts:

curl http://public-repo-1.hortonworks.com/

If the connection is not blocked then you should see something similar to this:

# curl http://public-repo-1.hortonworks.com/
<html><!-- -*- mode: javascript -*- -->
<head>
<title>HDP Repository</title><!--'-->
<style>
#current_dir {
  background-color: #eeeeee;
  padding: 8px;
}
</style>
<style type="text/css">
  H1 { text-align: center}
 </style>
 <script type="text/javascript">

var pwd = '';
var root;
var currentTree;
var trek = [];
var url = "http://s3.amazonaws.com/public-repo-1.hortonworks.com";
//var url = "http://s3.amazonaws.com/dev.hortonworks.com";
function makeTree(data) {
  var tree = {};
  for (var i = 0; i < data.length; i++) {
   var tokens = data[i].split("/");
   var branch = tree;
   for (var b = 0; b < tokens.length; b++) {
     var t = tokens[b];
     if ( !branch[t] ) {
       branch[t] = {};
     }
     branch = branch[t];
   }
  }
  return tree;
}

function backtrack() {
  currentTree = trek.pop();
  pwd = pwd.substring(0, pwd.lastIndexOf("/"));
  if (!currentTree) currentTree = root;
  getTree(currentTree);
}

function getTree(tree, branch) {
  if (branch) {
   currentTree = tree[branch];
   trek.push(tree);
   pwd += "/" + branch;
  }
  var display = '<div id="current_dir"><b>' + pwd.substr(1) + '/</b>  ';
  display += "<br/><a style='color:green' href=\"javascript:backtrack()\">Parent Directory</a> <span style='color: #999'>(To Move Back To Repository)</span></div><br/>\n";
  display += displayTree(currentTree);

  document.getElementById("display").innerHTML = display;
  if (pwd == '') {
      document.getElementById('current_dir').style.display = 'none';
      document.getElementById('loading').innerHTML = '<h3><u>Repositories:</u></h3>';
      //document.getElementById('loading').innerHTML = '';
  } else {
      document.getElementById('current_dir').style.display = 'block';
      document.getElementById('loading').innerHTML = '';
  }
  window.location.hash = pwd;
}

function myescape(s) { return escape(s).replace(/\+/g, '%2b'); }
//function myescape(s) { return escape(s).replace('%27', '\\%27'); }
function escape_quotes(s) {
    return s.replace(/\'/g, '\\\'').replace(/\"/g, '\\\"'); }
function evalable(s) {
    return "'" + escape_quotes(s) + "'"; }
    //return escape_quotes("'" + s + "'"); }
function displayTree(tree) {
  var display = '';
  for (var b in tree) {
   if (tree.hasOwnProperty(b)) {
       var link;
     if (isLeaf(tree[b])) {
         link = '<a href="' + url + myescape(pwd + "/" + b) + '">' + b + "</i><br/>\n";
     } else {
       link = "<a style=\"font-weight:bold;\" href=\"javascript:getTree(currentTree, " + evalable(b) + ")\">" + b + "</a><br/>\n";
     }
     //console.log('Adding link', link);
     display += link;
   }
  }
  return display;
}

function isLeaf(b) {
  for (var l in b) {
    if (!b.hasOwnProperty[l]) return false;
  }
  return true;
}

function parse(xmlDoc) {
  var data = [];
  var keys = xmlDoc.getElementsByTagName("Key");
  for (var i = 0; i < keys.length; i++) {
    var e = keys[i].firstChild.nodeValue;
    
    // only interested in some folders
    if (e.indexOf("/./") < 0
        && e.search(/_\$folder\$/) == -1
        && e.search(/.DS_store/) == -1)
      data.push(e);
  }  
  return data;
}

function jump_to_current_hash() {
    var hash = window.location.hash;//'#Soft Moon/The Soft Moon'
    hash = hash.substring(1);
    
    // Nevermind, that hash gets escaped in firefox but not safari/ie
    hash = (location.href.split("#")[1] || "");
    hash = unescape(hash)

    var dirs = hash.split('/');
    for (var i=0; i<dirs.length; i++) {
        //console.log('Descending into', dirs[i]);
        getTree(currentTree, dirs[i] == '' ? undefined : dirs[i]);
    }
}

function init(){
    var repo_paths = [];
    function fetch_index(starting_from) {
        var request = window.XMLHttpRequest
            ? new XMLHttpRequest()
            : new ActiveXObject("Microsoft.XMLHTTP");

        var suffix = starting_from ? '&marker=' + starting_from : '';
        request.open("GET", url + '?max-keys=300' + suffix, true);

        request.onreadystatechange = function(){
            if (request.readyState == 4 && request.status == 200) {
                if (request.responseXML){
                    var last_page = (request.responseXML
                                     .getElementsByTagName('IsTruncated')[0]
                                     .childNodes[0].nodeValue == 'false');

                    // Add these new repos to our running list
                    var new_repo_paths = parse(request.responseXML);
                    for (var i=1; i<new_repo_paths.length; i++)
                        // Skip the first one (i==0), cause it's just
                        // the "marker" (in amazon api) we used to get
                        // here.  Put the rest into the repos queue.
                        repo_paths.push(new_repo_paths[i]);

                    if (last_page) {
                        // This is the last one!  Let's rock!
                        root = makeTree(repo_paths);
                        currentTree = root;
                        console.log('Jumping to hash');
                        jump_to_current_hash();
                        console.log('Jumped to hash');
                        // Hide the loading indicators
                        //document.getElementById('loading').style.display = 'none';
                    } else {
                        // Get another page
                        //document.getElementById('loading').innerHTML =
                            //document.getElementById('loading').innerHTML 
                            //+ 'Please wait loading loading the page....';
                        document.getElementById('loading').innerHTML = 'Please wait loading the page....';
                        fetch_index(repo_paths[repo_paths.length-1]);
                    }
                }
            }
        }
        request.send(null);
    }

    fetch_index();
}


// Because IE
if(Function.prototype.bind&&console&&typeof console.log=="object"){["log","info","warn","error","assert","dir","clear","profile","profileEnd"].forEach(function(method){console[method]=this.call(console[method],console);},Function.prototype.bind);}
if(!window.log){window.log=function(){log.history=log.history||[];log.history.push(arguments);if(typeof console!='undefined'&&typeof console.log=='function'){if(window.opera){var i=0;while(i<arguments.length){console.log("Item "+(i+1)+":"+arguments[i]);i++;}}
else if((Array.prototype.slice.call(arguments)).length==1&&typeof Array.prototype.slice.call(arguments)[0]=='string'){console.log((Array.prototype.slice.call(arguments)).toString());}
else{console.log(Array.prototype.slice.call(arguments));}}
else if(!Function.prototype.bind&&typeof console!='undefined'&&typeof console.log=='object'){Function.prototype.call.call(console.log,console,Array.prototype.slice.call(arguments));}
else{if(!document.getElementById('firebug-lite')){var script=document.createElement('script');script.type="text/javascript";script.id='firebug-lite';script.src='https://getfirebug.com/firebug-lite.js';document.getElementsByTagName('HEAD')[0].appendChild(script);setTimeout(function(){log(Array.prototype.slice.call(arguments));},2000);}
else{setTimeout(function(){log(Array.prototype.slice.call(arguments));},500);}}}}

</script>
</head>

<body onload="init();">

<!--<h2 align="left"><a href="http://s3.amazonaws.com/dev.hortonworks.com/dir_list1.html">Repository Home</a></h2>
<h2 align="right"><a href="http://hortonworks.com">HortonWorks</a></h2>-->


<DIV>
<!--<SPAN STYLE="float:left"><h2 align="left"><a href="http://s3.amazonaws.com/dev.hortonworks.com/index.html">Repository Home</a></h2></SPAN>-->
<SPAN STYLE="float:left"><h2 align="left"><a href="http://s3.amazonaws.com/public-repo-1.hortonworks.com/index.html">Repository Home</a></h2></SPAN>
<SPAN STYLE="float:right"><h2 align="right"><a href="http://hortonworks.com">HortonWorks</a></h2></SPAN> 
<DIV STYLE="clear:both"></DIV>
</DIV>
<H1><u>!!! HortonWorks Data Platform Release Repository !!!</u></H1>
<div style="font-family: courier, courier new;" id="loading"></div>
<p style="display: none;"><b>:</b><span id="pwd"></span></p>
<div id="display"></div>
</body>
</html>

Re: HDP Stack Repositories not found

Expert Contributor

you can run this command as well to test the connection:

# telnet public-repo-1.hortonworks.com 80
Trying 52.84.13.149...
Connected to public-repo-1.hortonworks.com.



Re: HDP Stack Repositories not found

New Contributor

Hello @Manish Gupta! I realize this thread is a year old, but I am having the same problem and was hoping you might be able to help me as well. The chosen answer (the curl command) resulted in the stack for me unlike @Ahmad Debbas and therefore it did not reveal the solution to my problem. The telnet connection as you suggested above, however, timed out and did not connect. Does this mean there is still a problem with my connection? I am also connected to the internet, as I was able to download the server in the same manner.

I would really appreciate any ideas you may have as to what the problem is. There is a more verbose description of my problem in my thread/question here, however it is almost identical to @Ahmad Debbas's original question.

https://community.hortonworks.com/questions/45147/ambari-setup-select-stack-404-error.html

Re: HDP Stack Repositories not found

Expert Contributor

Hello Savanna

I ended up setting a local HDP repository as described in the link below. I was never able to solve the error messages i originally reported in this thread (generated by online repositories). Local repositories turned out to be very useful as i didn't have to download HDP repositories again everytime i installed an HDP cluster ( i did several installations before i reached a healthy cluster)

http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_Installing_HDP_AMB/content/_using_a_local...

hope this helps

Regards

Ahmad

Re: HDP Stack Repositories not found

New Contributor

Hello Ahmad,

Thank you very much for your response. I will do this as well, using your link. I may have to install multiple clusters as well so thanks for the tip :)

Kind regards,

Savanna

Don't have an account?
Coming from Hortonworks? Activate your account here