question Re: cloudera fails to resolve hosts in Archives of Support Questions (Read Only)

cloudera fails to resolve hosts

umb — Fri, 16 Sep 2022 09:12:11 GMT

Initially, i set up 2 machines (Ubuntu 12.04, x64) on vSphere server.

The name and ip of these two machines were

host                  ip
vm-cluster-node1      10.211.55.100
vm-cluster-node2      10.211.55.101

I have installed cloudera manager in vm-cluster-node1.

Then i cloned second one (vm-cluster-node2) to create 2 more hosts, and changed the ip and names as:

host                  ip
vm-cluster-node3      10.211.55.102
vm-cluster-node4      10.211.55.103

But the problem is, when i add these all 4 machines from cloudera, no matter how many times i try, i can only see two machines in hosts tab. later i realized that, if i refresh the web-page, i can see 2 machines only, but the second machine is switched between vm-cluster-node2, vm-cluster-node3 and vm-cluster-node4.

to illustrate, i have included images to make things clear.

So, as far i've understood, the cloudera manager is not able to resolve the hosts cloned from same source as different machines. Even though the host-names and IPs have been changed. So is there anything that is common in these machines and this problem is occurring?

Re: cloudera fails to resolve hosts

Grizzly — Fri, 07 Nov 2014 18:07:04 GMT

The networking and security requirements for installation are listed here, make sure you have read through and configured based on this discussion.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_cm_requirements.html#cmig_topic_4_3_3_unique_1

each host must be able to perform forward and reverese lookup of the others.

if /etc/hosts is how you are managing dns, make sure they reflec the proper IP/Hostname

The other thing to verify is that your cloned vm's have unique MAC addresses for their network interfaces, you should be able to look at the physical (MAC) address on each node with the ifconfig command.

Re: cloudera fails to resolve hosts

umb — Mon, 10 Nov 2014 07:29:26 GMT

Thanks for the reply.

When i checked, i found the machines have different MAC addresses, but reverse DNS is not working.

However, i have found that the HOST_ID for the three hosts are same. Is this because i cloned the vms from same source. Could this be the reason for problem. If yes, how can i change the host-id of the hosts?

Re: cloudera fails to resolve hosts

Grizzly — Tue, 11 Nov 2014 22:39:47 GMT

It could be that the hostID is what is causing the problem. Changing the hostid value on your hosts will be something you need to referfence from your linux distribution's documentation.

I see references in our CM code where if we do not find a hostid, that the hostname is used instead, so having duplicate hostid's is probably creating issues.

Tood

Re: cloudera fails to resolve hosts

umb — Wed, 12 Nov 2014 05:47:45 GMT

Yes that was the proble,

Tgrayson wrote:
It could be that the hostID is what is causing the problem. Changing the hostid value on your hosts will be something you need to referfence from your linux distribution's documentation.

I see references in our CM code where if we do not find a hostid, that the hostname is used instead, so having duplicate hostid's is probably creating issues.

Tood

i did that.... i changed host id by modifingfollowing file:

/etc/hosts/cloudera-scm-agent

by adding

CMF_AGENT_ARGS="--host_id new_host_id"

Re: cloudera fails to resolve hosts

ysk — Mon, 27 Jun 2016 12:05:01 GMT

Hi,

We have the same problem i.e. when we use 2 clones of the same VM and try to connect the Master node to them, only one of them is visible at a time. We have gone through the thread above and made sure that -

i) Hostnames of the 2 VMs are different

ii) Hostids of the 2 VMs are different

iii) /etc/hosts has names and IPs of all 3 machines i.e. Master, Node1 and Node2

iv) Physical addresses of the 2 VMs is different

However, the problem persists. Any further inputs into what could be causing the issue please? Any help would be appreciated.

Thanks in advance.

Regards,

Yogesh

Re: cloudera fails to resolve hosts

bgooley — Mon, 27 Jun 2016 15:29:33 GMT

Hi Yogesh,

Each agent will identify the host to Cloudera Manager with a "uuid" that is set by the agent. So, if you have two hosts with the same uuid (which I assume could occur when cloning), Cloudera Manager would see them as the same host.

To verify this is the cause, look at the /var/lib/cloudera-scm-agent/uuid file on each host.

If the uuid value is the same for any hosts, find the hosts that DO NOT appear in Cloudera Manager and do this on the host that is not appearing in CM:

(1)

service cloudera-scm-agent stop

(2)

rm /var/lib/cloudera-scm-agent/uuid

(3)

service cloudera-scm-agent start

When the agent starts, it should generate a new unique uuid and when it heartbeats to Cloudera Manager, CM should now detect that it is a new host.

Re: cloudera fails to resolve hosts

ysk — Tue, 28 Jun 2016 04:31:55 GMT

This is fantastic Bgooley! Your solution worked.

The cloned VM indeed had the same UUID as the parent. As per your instructions, we stopped the scm agent, deleted the uuid and restarted scm agent. We then added the node to the cluster via Cloudera Manager and it got registered correctly. We can now see both the VM nodes.

Thank you so much! Appreciate your help.