Support Questions
Find answers, ask questions, and share your expertise

Install problems with HDP 2.6

Expert Contributor

Hi. I am at the install stage for my HDP2.6 cluster, for Ubuntu14 and Ambari 2.5.0.3. My master node fails with:

E: Unable to locate package accumulo-2-4-2-0-258

My s1 node fails (warning):

E: Unable to locate package hdp-select
2017-05-11 19:34:05,196 - Failed to install package hdp-select. Executing '/usr/bin/apt-get update -qq'
2017-05-11 19:34:47,796 - Retrying to install package hdp-select after 30 seconds
Command aborted. Reason: 'Server considered task failed and automatically aborted it'

My s2 node fails (warning):

2017-05-11 19:33:59,355 - Installing package hadoop-2-6-0-3-8-client ('/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install hadoop-2-6-0-3-8-client')
Command aborted. Reason: 'Server considered task failed and automatically aborted it'

My s2 node fails (warning):

2017-05-11 19:34:06,067 - Installing package accumulo-2-6-0-3-8 ('/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install accumulo-2-6-0-3-8')
Command aborted. Reason: 'Server considered task failed and automatically aborted it'

These are all the first install tasks.

Are these related? How might I fix them?

TIA!

UPDATE:

I am now getting

An internal system exception occurred: Base url http://public-repo-1.hortonworks.com/HDP/sles12/2.x/updates/2.6.0.3 is already defined for another repository version. Setting up base urls that contain the same versions of components will cause stack upgrade to fail.

error. I have not changed anything since my original post except install tree.

19 REPLIES 19

Expert Contributor

Which repository are you currently using, while try to install ?

HDP, OS ?

output :

yum repolist

also provide the full log it will help to analyze as quick as possible...

Expert Contributor

@Palanivelrajan Chellakutty Thanks, my log is here (I tried again just now, so the entries at the end 12/5/17 would be teh correct ones). I am using Ubuntu14, HDP2.6, how do I check which repos I am using for the install?

Explorer

I need the ambari-server.log to analyze this issues.

Expert Contributor

@Peter Kim Thanks, my log is here (I tried again just now, so the entries at the end 12/5/17 would be the correct ones).

Something is wrong with your repositories, or with your Internet connection if you use repos on the Internet. Try to install just one of those packages directly from the command line, apt-get will show you more detailed error than Ambari, and then try to fix whatever is needed until apt-get install works, for example

$ apt-get install hadoop-2-6-0-3-8-client

Prepend "sudo" if needed. Also try to install a package unrelated to Hadoop like "tree": "sudo apt-get install tree". After both commands work, you can go back to Ambari and try to re-install the cluster.

Expert Contributor

If I do apt-get install hadoop-2-6-0-3-8-client it seems to be OK. I do not answer yes so as to not mess things up. tree installed OK. i have just noticed my cluster connection keeps dropping, could this be the problem?

Expert Contributor

ambari-server log

----------------------------------

08:56:15,539 INFO [Stack Version Loading Thread] LatestRepoCallable:219 - Stack HDP-2.3 cannot resolve OS debian6 to the supported ones: suse11,redhat7,debian7,redhat6,ubuntu14,ubuntu12. Family: null 07 May 2017 08:56:15,676 INFO [Stack Version Loading Thread] LatestRepoCallable:80 - Loading latest URL info for stack HDP-2.4 from http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json 07 May 2017 08:56:15,761 INFO [Stack Version Loading Thread] LatestRepoCallable:219 - Stack HDP-2.4 cannot resolve OS debian6 to the supported ones: suse11,redhat7,debian7,redhat6,ubuntu14,ubuntu12. Family: null 07 May 2017 08:56:15,893 INFO [Stack Version Loading Thread] LatestRepoCallable:80 - Loading latest URL info for stack HDP-2.5 from http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json 07 May 2017 08:56:16,039 ERROR [main] ClusterImpl:465 - Can not get service info: stackName=HDP, stackVersion=2.4, serviceName=ZEPPELIN 07 May 2017 08:56:16,473 WARN [Stack Version Loading Thread] LatestRepoCallable:237 - Could not load version definition for HDP-2.5 identified by http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.5.3.0/HDP-2.5.3.0-37.xml. http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.5.3.0/HDP-2.5.3.0-37.xml java.io.FileNotFoundException: http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.5.3.0/HDP-2.5.3.0-37.xml at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1872) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at java.net.URL.openStream(URL.java:1045) at org.apache.ambari.server.state.repository.VersionDefinitionXml.load(VersionDefinitionXml.java:298) at org.apache.ambari.server.state.stack.LatestRepoCallable.mergeDefinitions

================================

-current repo settings set for public url.

you can find the repo file here ...../etc/yum.repos.d/

==> Need below output, to review all basic settings in place...

cat <ambari repo file>

cat <HDP repo files>

yum repolist

Expert Contributor

your current repo file looking into below public url below, no longer available in HDP repo directory.

http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.5.3.0/HDP-2.5.3.0-37.xml

Below are the HDP latest version available for ubuntu16 OS...Please go ahead and update your repo file below HDP version it will work.

HDP/ubuntu16/2.x/updates/

Parent Directory

(To Move Back To Repository)

2.5.5.0

2.6.0.3

Expert Contributor

Please go ahead below, since your are using "ubuntu14 OS", modify/replace your repofile ubuntu14 instead of ubuntu16.

http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.3.0/HDP-2.5.3.0-37.xml

HDP/ubuntu14/2.x/updates/2.5.3.0/ Parent Directory (To Move Back To Repository) HDP-2.5.3.0-37.xml HDP-2.5.3.0-ubuntu14-deb.tar.gz HDP-2.5.3.0-ubuntu14-deb.tar.gz.asc HDP-2.5.3.0-ubuntu14-deb.tar.gz.md5 HDP-2.5.3.0-ubuntu14-m2-artifacts.tar HDP-2.5.3.0-ubuntu14-tars-tarball.tar.gz HDP-2.5.3.0-ubuntu14-tars-tarball.tar.gz.asc HDP-2.5.3.0-ubuntu14-tars-tarball.tar.gz.md5 build.id build_metadata.txt db dists hdp.list pool relocatable slider-app-packages tars

Expert Contributor

Thanks @Palanivelrajan Chellakutty. Could you very briefly tell me how to do that? Is it the first step in the Ambari install where it asks for the base URLs and do I also have to add it to Ubuntu somehow?

Expert Contributor

need below output where you try to install ambari-server on host machine

Output needed:

cat /etc/yum.repos.d/ambari.repo

cat /etc/yum.repos.d/HDP.repo

cat /etc/yum.repos.d/HDP-UTILS.repo

yum repolist

Expert Contributor

I am now getting

An internal system exception occurred: Base url http://public-repo-1.hortonworks.com/HDP/sles12/2.x/updates/2.6.0.3 is already defined for another repository version. Setting up base urls that contain the same versions of components will cause stack upgrade to fail.

error. I have not changed anything since my original post except install tree.

Explorer

Check List.

1. Check Ambari-Server DB

# ambari-server check-database

You must return a message "No errors were found."

If you installed hdp&ambari normally, then Ambari-Server will connect in ambari database of your postgres.

But I think, maybe you were upgraded twice between hdp 2.4.0 -> hdp 2.5.3.0 and hdp 2.5.3.0-> hdp 2.6.0.0 of rolling or manual upgrade.

So I asking, did you upgrade clearly from previous hdp version?

Now, your ambari database in postgres is conflicting cluster, service, some tables, FK constraint(alert_definition table doesn't exist.)..

2. Check Ambari DDL Postgres.

# cd /var/lib/ambari-server/resources

If embeded postgres

# vim Ambari-DDL-Postgres-EMBEDDED-CREATE.sql

Else if custom postgres

# vim Ambari-DDL-Postgres-CREATE.sql

ambari-server still can't find you hadoop cluster 'cresta'.

3. Check ambari database in Postgres.

connect postgres.

\connect ambari

SELECT * FROM ambari.alert_definition;

SELECT * FROM ambari.alert_history;

SELECT * FROM alert_definition WHERE definition_id=74;

SELECT * FROM alert_history WHERE alert_definition_id=74;

definition_id in alert_definition table is alert_definition_id in alert_history table like FK.

But alert_definition_id (74) does not exist in alert_definition table.

It seems something wrong and not clean upgraded.

Expert Contributor

No errors on ambari-server check-database. I am doing a fresh install (I deleted an old version).

Not sure about the Postgres stuff, how do I use it and how do I know if I am already using it?

Explorer

Execute these commans on installed ambari-server node.

vim /etc/ambari-server/conf/ambari.properties

server.jdbc.database=mysql

server.jdbc.database_name=ambari

If you are using postgres database, then server.jdbc.database values is postgres.

How did you delete an older version?

Expert Contributor

it looks new error show up you are trying for upgrade...please confirm what you doing currently

Question :

1. are you trying ambari-server fresh install ?

2. or trying to do ambari-server upgrade from one version to newer version ?

Output needed:

cat /etc/yum.repos.d/ambari.repo

cat /etc/yum.repos.d/HDP.repo

cat /etc/yum.repos.d/HDP-UTILS.repo

yum repolist

Expert Contributor

Fresh install.

I am using Ubuntu, hopefully these are what you are after:

ed@master:~$ cat /etc/apt/sources.list.d/ambari.list
#VERSION_NUMBER=2.5.0.3-7
deb http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.5.0.3 Ambari main
e
ed@master:~$ cat /etc/apt/sources.list.d/HDP.list
deb http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.6.0.3 HDP 
ed@master:~$ cat /etc/apt/sources.list.d/HDP-UTILS.list 
deb http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu12 HDP-UTILS mained@master:~$ 
ed@master:~$ cat /etc/apt/sources.list
# deb cdrom:[Ubuntu 14.04.4 LTS _Trusty Tahr_ - Release amd64 (20160217.1)]/ trusty main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://gb.archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ trusty main restricted
## Major bug fix updates produced after the final release of the
## distribution.
deb http://gb.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://gb.archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ trusty universe
deb http://gb.archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://gb.archive.ubuntu.com/ubuntu/ trusty-updates universe
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu 
## team, and may not be under a free licence. Please satisfy yourself as to 
## your rights to use the software. Also, please note that software in 
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://gb.archive.ubuntu.com/ubuntu/ trusty multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ trusty multiverse
deb http://gb.archive.ubuntu.com/ubuntu/ trusty-updates multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ trusty-updates multiverse
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://gb.archive.ubuntu.com/ubuntu/ trusty-backports main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ trusty-backports main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu trusty-security main restricted
deb-src http://security.ubuntu.com/ubuntu trusty-security main restricted
deb http://security.ubuntu.com/ubuntu trusty-security universe
deb-src http://security.ubuntu.com/ubuntu trusty-security universe
deb http://security.ubuntu.com/ubuntu trusty-security multiverse
deb-src http://security.ubuntu.com/ubuntu trusty-security multiverse
## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu trusty partner
# deb-src http://archive.canonical.com/ubuntu trusty partner
## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
deb http://extras.ubuntu.com/ubuntu trusty main
deb-src http://extras.ubuntu.com/ubuntu trusty main

Expert Contributor

I have done replica your issue on OS ubuntu16, don't see any issue while installing ambari-server/agent and HDP...Please confirm which step you got an error while try to install ambari server ?

OS Version : ubuntu16 Ambari : 2.5.0.3 HDP : 2.5.5.0 repository used : public url cd /etc/apt/sources.list.d ==> Download software wget -nv http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.5.0.3/ambari.list -O /etc/apt/sources.list.d/ambari.list apt-key adv --recv-keys --keyserver keyserver.ubuntu.com B9733A7A07513CAD apt-get update testhdp@testhdp:/etc/apt/sources.list.d$ cat ambari.list #VERSION_NUMBER=2.5.0.3-7 deb http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.5.0.3 Ambari main ==> Confirm that Ambari packages downloaded successfully by checking the package name list. apt-cache showpkg ambari-server apt-cache showpkg ambari-agent apt-cache showpkg ambari-metrics-assembly ==> Install the Ambari bits. This also installs the default PostgreSQL Ambari database. apt-get install ambari-server apt-get install ambari-agent

Expert Contributor

Please ignore previous notes, same notes attached for your reference.

I have done replica your issue on OS ubuntu16, don't see any issue while installing ambari-server/agent and HDP...Please confirm which step you got an error while try to install ambari server ?

; ;