Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Deploying nodes from quickstart VM?

avatar
Contributor

Hi,

I've got the quickstart vm up and running and working pretty well and I'd like to test out a few of the Cloudera administrative tasks using manager.

 

I'm wondering if anyone has successfully deployed additional nodes from the quickstart VM and gotten a cluster (virtual or physical) up and running?

 

Thanks

Ty

1 ACCEPTED SOLUTION

avatar
Guru
If you want to have a virtual cluster, I would strongly recommend just
starting with vanilla Linux VMs, downloading and running the Cloudera
Manager installer on one of them, and building a new cluster. You may be
surprised at how easy it is, once you have the VMs networked together
properly. You would have to reset so much on the QuickStart VM to get it to
incorporate a copy of itself as another node in the cluster - it would
actually be harder than starting from scratch. The QuickStart VM is
designed to "just work" as robustly as possible regardless of how the
virtual network is setup, and that requires that it make some assumptions
that it is just a single node. So be aware that you're going to run into
some issues if you try this, and we do not try to cater to this use case.

Specifically, you're going to run into a lot of networking issues. The VM
has the hostname quickstart.cloudera 'baked' into it. To add another node,
you would need another hostname, and that's going to require changing so
many config files and resetting so many services that you would basically
be starting from scratch anyway. You would also need to be careful with IP
addresses. If another network device is not available early enough in the
boot, the VM will use 127.0.0.1 - which works fine as a single-node, but
that's not how you want machines to refer to themselves in a distributed
system, because as soon as it's resolved elsewhere it's wrong. So you'd
need to make sure the VM had an externally routeable IP (e.g. use bridged
networking, or a similar option) and was rebooted (in my experience, you
have to reboot twice after making the change) in order to have the correct
networking device be available early enough in the boot process. Not to
mention, this is all in theory - I don't know that anyone has successfully
done this. Again - it's so much easier to just install using Cloudera
Manager on top of some new Linux VMs.


View solution in original post

2 REPLIES 2

avatar
Guru
If you want to have a virtual cluster, I would strongly recommend just
starting with vanilla Linux VMs, downloading and running the Cloudera
Manager installer on one of them, and building a new cluster. You may be
surprised at how easy it is, once you have the VMs networked together
properly. You would have to reset so much on the QuickStart VM to get it to
incorporate a copy of itself as another node in the cluster - it would
actually be harder than starting from scratch. The QuickStart VM is
designed to "just work" as robustly as possible regardless of how the
virtual network is setup, and that requires that it make some assumptions
that it is just a single node. So be aware that you're going to run into
some issues if you try this, and we do not try to cater to this use case.

Specifically, you're going to run into a lot of networking issues. The VM
has the hostname quickstart.cloudera 'baked' into it. To add another node,
you would need another hostname, and that's going to require changing so
many config files and resetting so many services that you would basically
be starting from scratch anyway. You would also need to be careful with IP
addresses. If another network device is not available early enough in the
boot, the VM will use 127.0.0.1 - which works fine as a single-node, but
that's not how you want machines to refer to themselves in a distributed
system, because as soon as it's resolved elsewhere it's wrong. So you'd
need to make sure the VM had an externally routeable IP (e.g. use bridged
networking, or a similar option) and was rebooted (in my experience, you
have to reboot twice after making the change) in order to have the correct
networking device be available early enough in the boot process. Not to
mention, this is all in theory - I don't know that anyone has successfully
done this. Again - it's so much easier to just install using Cloudera
Manager on top of some new Linux VMs.


avatar
Contributor

Thanks Sean.  I appreciate the guidance.

 

I now have Vmware ESXi setup and an ubuntu 14.04 server guest.  I'm going through "Installation Path A - Automted Installation by Cloudera Manager".  All going well so far.