Ambari setup stuck at step 3

[#1]

Hi,

While installing Ambari 2.6 to setup a 2 node HDP cluster i was stuck at step 3 saying "Please wait while the hosts are being checked for potential problems..". i have checked the ambari-server.log and it says "Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1"

Here is the excerpt from the ambari-server.log

13 Nov 2017 13:09:24,422 WARN [ambari-action-scheduler] ExecutionCommandWrapper:225 - Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1 13 Nov 2017 13:09:25,427 WARN [ambari-action-scheduler] ExecutionCommandWrapper:225 - Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1 13 Nov 2017 13:11:11,433 INFO [pool-18-thread-1] MetricsServiceImpl:65 - Attempting to initialize metrics sink 13 Nov 2017 13:11:11,434 INFO [pool-18-thread-1] MetricsServiceImpl:81 - ********* Configuring Metric Sink ********** 13 Nov 2017 13:11:11,434 INFO [pool-18-thread-1] AmbariMetricSinkImpl:95 - No clusters configured. 13 Nov 2017 13:16:11,434 INFO [pool-18-thread-1] MetricsServiceImpl:65 - Attempting to initialize metrics sink 13 Nov 2017 13:16:11,435 INFO [pool-18-thread-1] MetricsServiceImpl:81 - ********* Configuring Metric Sink ********** 13 Nov 2017 13:16:11,435 INFO [pool-18-thread-1] AmbariMetricSinkImpl:95 - No clusters configured.

pls help

i have followed the steps in hortonworks youtube channel on setting up the cluster but dont know what i am missing.

Thanks

Rahul

[#2]

As we see that the folllowing message is appearing in your logs/UI. Which indicates that the hosts are not yet successfully registered.

 Cluster not found, clusterName=clusterID=-1

In this step ambari server basically performs some of the checks like "transparent huge pages", "ntpd service", "SELinux" ...etc So please check if you have setup the passwordless ssh properly from ambari server host to the agent host.

Also please check if the "iptables" is disabled on the agent machine (sothat it can communicate with ambari server on default ports 8440, 8441). So please check the following from ambari agent machine. Please check the ambari-agent "/etc/hosts" file if it is able to resolving the ambari server hostname.

# nc -v $AMBARI_HOSTNAME 8440
# nc -v $AMBARI_HOSTNAME 8441
(OR)
# telnet $AMBARI_HOSTNAME 8440
# telnet $AMBARI_HOSTNAME 8441

As this is a fresh cluster setup hence i will suggest if above does not solve the problem then try to reset the ambari DB by issuing the following command and then retry the ambari-server (cluster creation wizard)

# ambari-server stop
# ambari-server reset           ----->   (this will clean up ambari DB to default state)
# ambari-server start

.

[#3]

Thanks @Jay Kumar SenSharma for your quick response

As u said all the host entries , ntpd , selinux all set up properly but still the same issue persists.

i am able to reach the ambari server host from agent at 8440 and 8441

i have also done the below steps several times but still same "Unable to lookup the cluster by ID" persists.

1. # ambari-server stop
2. # ambari-server reset -----> (this will clean up ambari DB to default state)
3. # ambari-server start

i am attaching the ambari-server.log ambari-server.txt

Also attaching the Install wizard screenshot ambari.png

Thanks

Rahul

[#4]

@Rahul Narayanan

In your ambari-server.log i see the following message:

ERROR [main] StackContext:217 - Could not load repo results
java.net.UnknownHostException: public-repo-1.hortonworks.com

.

Which makes me feel that your ambari-server host does not have internet connectivity Or there is a firewall rule Or proxy that is restricting Ambari Server host to pull repo data from the Hortonworks public repo. I am suspecting that the same might be the case from Ambari Agent hosts as well and hence after validation during host registration when ambari is tryign to add the hosts it is getting stuck.

Please check if your ambari agent hosts have internet connectivity or access to the public repo?

Additionally, Please check that all the hosts are having "lowercase" hostnames, Mixed or Upper case hostnames sometimes causes the issue during installation as well as with HDFS service checks.

.

[#5]

Additionally, If you are using Local Repositories Or using Http Proxy Host then please do not forget to specify the following properties as described in the following doc:

-Dhttp.proxyHost=<yourProxyHost> -Dhttp.proxyPort=<yourProxyPort><br>

.

Also in the "/etc/yum.conf" of every host:

proxy=http://<yourProxyHost>:<yourProxyPort>

https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-administration/content/ch_setting...

.

[#6]

@Jay Kumar SenSharma

I am using local repository as i don't have internet connection on my nodes. I am giving the local repo details during step 1 (select versions) for hdp and hdputils.

i have my host file will nodes fqdn in lowecase

Thanks

Rahul

[#7]

Can you please check , From your ambari-agent hosts are you able to access the Local Repository?

[#8]

yes, i am able to do wget on to my local repository from ambari agent as well also i tried yum install to check and it was able to connect to local repository.

[#9]

@Jay Kumar SenSharma yes am bale to access the local repository from ambari-agent as well

Also the error "15 Nov 2017 15:00:56,033 ERROR [main] StackContext:217 - Could not load repo results java.net.UnknownHostException: public-repo-1.hortonworks.com"

comes immediately once i start my ambari-server. where will i configure ambari to look in to my local repo and not the public repo

[#10]

@Rahul Narayanan

These are the 2 files (baseurl)you should update to point to the local repo

/etc/yum.repos.d/ambari.repo
/etc/yum.repos.d/hdp.repo

Hope that helps