Created on
02-05-2019
02:36 AM
- last edited on
10-24-2019
06:23 PM
by
ask_bill_brooks
At wits end wit this hortwork work platform , trying to get it kerborized.
The phase where it Testing Kerberos clients, keep getting this error that it cannot find the ambari host and displays the fqdn
The ambari host is name resolved by 2 hosted domains internally with no problem. ie ambari.example.com and ambari.corp.example.com. I also added this in /etc/hosts files on each server in cluster including ambari server , each maser node , slave node and DC on our network to eliminate DNS as an issue ( which I cannot believe it is as at teh command line these things can resolve all day long to each other) What logs can I examine to give more information on this error please?
var/log/ambari-server is not much help
Created 02-05-2019 02:43 AM
If you define two hostname s inside the "/etc/hosts" file then you might find issues in kerberized environment
Example:
192.xx.yy.170 ambari.example.com ambari.corp.example.com
.
In thew above case we might see that the kerberos clients resolves the incorrect hsotname. So can you please keep only one host there and then try again.
Also can you please share the complete message where you see "Host not found, hostname" that way we can check the exact code base to understand in which conditions it might show that message
Created 02-05-2019 10:18 PM
Hi Jay , thanks for reply.
I changed /etc/hosts on ambari to reflect your suggestion ( I have not done all in cluster as yet)
10.x.x.x ambari.example.com
trie to enable , still returns error on creating principles for ambari ....the error from /var/log/ambari-server/ambari-server.log is below. ( is there anyplace else to look?)
2019-02-05 02:23:07,223 WARN [Server Action Executor Worker 1760] ServerActionExecutor:471 - Task #1760 failed to complete execution due to thrown exception: org.apache.ambari.server.HostNotFoundException:Host not found, hostname=ambari.example.com org.apache.ambari.server.HostNotFoundException: Host not found, hostname=ambari.example.com 2019-02-05 19:29:02,615 WARN [Server Action Executor Worker 1801] ServerActionExecutor:471 - Task #1801 failed to complete execution due to thrown exception: org.apache.ambari.server.HostNotFoundException:Host not found, hostname=ambari.example.com org.apache.ambari.server.HostNotFoundException: Host not found, hostname=ambari.example.com
Created 02-05-2019 10:18 PM
Does the Amabri server have to be "in" the cluster?
Created 02-05-2019 10:23 PM
2019-02-05 02:23:07,223 WARN [Server Action Executor Worker 1760] ServerActionExecutor:471 - Task #1760 failed to complete execution due to thrown exception: org.apache.ambari.server.HostNotFoundException:Host not found, hostname=ambari.example.com org.apache.ambari.server.HostNotFoundException: Host not found, hostname=ambari.example.com 2019-02-05 19:29:02,615 WARN [Server Action Executor Worker 1801] ServerActionExecutor:471 - Task #1801 failed to complete execution due to thrown exception: org.apache.ambari.server.HostNotFoundException:Host not found, hostname=ambari.example.com org.apache.ambari.server.HostNotFoundException: Host not found, hostname=ambari.example.com
Created 02-05-2019 10:23 PM
Hi Jay thanks. Above is the error log from /var/log/ambari-server/ambari-server.log I added removed the corp/example/.cpom entries from /etc/hosts
Aside questions of anyone can answer...
: Is there any documentation to suggest that you cannot use ubuntu 18.04 as the server? every doc I read does not reference this version, seems to stop at ubuntu 12 or 14...
Does the ambari server need to be host member of cluster? ...
Created 02-05-2019 10:27 PM
Regarding Ubuntu-18 supportability... you can refer to : Support matrix and click on your version of Ambari to see which all OS it is tested and certified to work with.
https://supportmatrix.hortonworks.com/
.
Regarding other query "Does the ambari server need to be host member of cluster? ..."
No, it is not mandatory to make AmbariServer host to be a member of Cluster.
Created 02-05-2019 10:25 PM
Ideally the "/etc/hosts" file kept identical on all cluster nodes to avoid any kind of hostname confusions.
.
It is not mandatory to install Agent on the AmbariServer host. (that is optional)
.
Regarding the "HostNotFoundException" can you please share the output of the following API call to see the hostss (to find out if any conflict)
# curl -H "X-Requested-By: ambari" -u admin:admin http://$AMBARI_HOSTNAME:8080/api/v1/clusters/$CLUSTER_NAME/hosts?fields=Hosts/ip,Hosts/host_name
.
Created 02-05-2019 11:47 PM
ambari using ssl 8443 so used this otherwise connection refused.
root@ambari:~# curl -H "X-Requested-By: ambari" -u admin:admin http://ambari.example.com:8443/api/v1/clusters/cluster-name/hosts?fields=Hosts/ip,Hosts/host_name
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
Created 02-05-2019 11:47 PM
ambari using ssl 8443 so used this otherwise connection refused.
root@ambari:~# curl -H "X-Requested-By: ambari" -u admin:admin http://ambari.example.com:8443/api/v1/clusters/cluster-name/hosts?fields=Hosts/ip,Hosts/host_name
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
Created 02-05-2019 11:54 PM
You can also open the same URL from the Browser where you have logged in the Ambari UI as well.
http://ambari.example.com:8443/api/v1/clusters/cluster-name/hosts?fields=Hosts/ip,Hosts/host_name
.
(OR) You can use the following curl call to produce the JSON output to some file like "/tmp/hosts.json". Also pleas emake sure to use the correct cluster name in the same URL.
# curl -k -H "X-Requested-By: ambari" -u admin:admin "http://ambari.example.com:8443/api/v1/clusters/cluster-name/hosts?fields=Hosts/ip,Hosts/host_name" -o "/tmp/hosts.json"
.
Created 05-08-2019 06:28 AM
i am upgrading from HDP 2.6.2 to HDP 3.1, i am using Ambari 2.7.3.0 and currently stuck on (generate missing principal ), my cluster is kerberized, the machine hosting ambari server doesn't have ambari agent, and i am also getting the same error (host not found) for the ambari master machine , below is the exact error from ambari server log
2019-05-07 16:55:58,023 WARN [Server Action Executor Worker 3611] ServerActionExecutor:471 - Task #3611 failed to complete execution due to thrown exception: org.apache.ambari.server.HostNotFoundException:Host not found, hostname=MYHOSTNAME.com org.apache.ambari.server.HostNotFoundException: Host not found, hostname=MYHOSTNAME.com at org.apache.ambari.server.state.cluster.ClustersImpl.getHost(ClustersImpl.java:456) at org.apache.ambari.server.state.ConfigHelper.getEffectiveDesiredTags(ConfigHelper.java:189) at org.apache.ambari.server.state.ConfigHelper.getEffectiveDesiredTags(ConfigHelper.java:173) at org.apache.ambari.server.controller.AmbariManagementControllerImpl.findConfigurationTagsWithOverrides(AmbariManagementControllerImpl.java:2371) at sun.reflect.GeneratedMethodAccessor424.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:50) at com.sun.proxy.$Proxy131.findConfigurationTagsWithOverrides(Unknown Source) at org.apache.ambari.server.state.ConfigHelper.calculateExistingConfigurations(ConfigHelper.java:2158) at org.apache.ambari.server.controller.KerberosHelperImpl.calculateConfigurations(KerberosHelperImpl.java:1725) at org.apache.ambari.server.controller.KerberosHelperImpl.getActiveIdentities(KerberosHelperImpl.java:1800) at org.apache.ambari.server.serveraction.kerberos.KerberosServerAction.calculateServiceIdentities(KerberosServerAction.java:511) at org.apache.ambari.server.serveraction.kerberos.KerberosServerAction.processIdentities(KerberosServerAction.java:455) at org.apache.ambari.server.serveraction.kerberos.CreatePrincipalsServerAction.execute(CreatePrincipalsServerAction.java:92) at org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:550) at org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:466) at java.lang.Thread.run(Thread.java:745)
can you please suggest what to do?
Created on 10-24-2019 06:30 AM - edited 10-24-2019 07:26 AM
@jsensharma This issue has been observed in the internal lab cluster as well.
curl -H "X-Requested-By: ambari" -u admin:pnarayanasamy"http://c2237-node1.labs.support.hortonworks.com:8080/api/v1/clusters/c2237/hosts"
ambari server & kdc host c2237-node1.labs.support.hortonworks.com is not listed in the items array response.
/etc/hosts having only one fqdn for all the ip's.
Created 07-01-2020 04:21 PM
Hi all,
I had faced the same issue while kerberizing a HDP 3.1.0 cluster integrated with Iislon.
The Ambari serer is a built on a Postgres DB cluster, hence it was NOT in the hadoop cluster.
so it was NOT on the host list.
[root@hdpdb ~]# curl -H "X-Requested-By: ambari" -u admin:admin "http://hdpdb.gz.local:8080/api/v1/clusters/Panyu/hosts"
I had to add the ambari server from adding host wizard and just install the clients.
It worked around the "Host not found, hostname= xxxx" issue in my case.
Hope it will help, cheer!