Re: Hadoop cluster on my home network

monikersupreme · ‎09-01-2017

Could I use a Docker container (ex. https://community.hortonworks.com/repos/75668/a-multi-node-docker-cluster-platform-to-quickly-sp.html ) configured as master and data nodes on each of my Windows machines instead of and Ubuntu 16.04 image via Virtualbox?

monikersupreme · ‎08-24-2017

I'm a C++/C# programmer interested in Hadoop and development withing the Big Data ecosystem. In order to get started I am attempting to set up a training cluster on some commodity hardware I have idle on my home network: https://community.cloudera.com/t5/Hadoop-101-Training-Quickstart/Learning-Hadoop-can-I-set-up-a-training-cluster-at-home/td-p/58985 Any advice would be much appreciated!

monikersupreme · ‎08-19-2017

Thanks for your response! The latter two machines (Thinkpad and 12 core machine) will have native Ubuntu installed; since it is my intention to install HDP directly on these machines will I still need virtualbox for the 12 core? Also, would it be possible for me to get the cluster running using only the first three machines (2 VMs and Thinkpad) as these machines are already up and running - and "add" the 12 core to the cluster later on (once it is built)?

monikersupreme · ‎08-19-2017

Thanks for your response! I do indeed intend to run an Ubuntu VM from within my Windows 7 and 10 boxes - the latter two machines (laptop, 12 core) will have Ubuntu 16.04 LTS installed although the laptop will only have limited SSD space. I'd like to be able to use both the former VMs and the nodes I have running on native Ubuntu in the same cluster. So, just for clarification, your advice is NOT to use the HDP Sandbox image but rather set up my own Ubuntu VMs and install my HDP cluster on those instead?

monikersupreme · ‎08-18-2017

I want to set up a Hadoop cluster on my home network as a training exercise and need advice as to the configuration that best suits the hardware I have available. I would like to use a combination of both virtual (on Windows) and hardware nodes (Ubuntu) with approximately a terabyte of dedicated disk space for the filesystem. The machines I have (or will have) at my disposal are: 8 core / 32GB (dual processor) with 4TB RAID5 array running Windows 7 4 core / 16GB with 1TB RAID5 running Windows 10 2 core / 8GB Thinkpad running Ubuntu 16.04 (need to install Linux) 12 core / 32GB (dual processor) with 1TB RAID5 array running Ubuntu 16.04 (need to assemble hardware and install Linux) I would like to get the VMs up and running first; Will the Hortonworks sandbox VM allow me to create a multi-node cluster with both virtual and hardware nodes?

monikersupreme · ‎08-17-2017

(Apologies in advance if this is the wrong place to be asking this) I want to set up a Hadoop cluster on my home network as a training exercise and need advice as to the configuration that best suits the hardware I have available. I would like to use a combination of both virtual (on Windows) and hardware nodes (Ubuntu) with approximately a terabyte of dedicated disk space for the filesystem. The machines I have (or will have) at my disposal are: 8 core / 32GB (dual processor) with 4TB RAID5 array running Windows 7 4 core / 16GB with 1TB RAID5 running Windows 10 2 core / 8GB Thinkpad running Ubuntu 16.04 (need to install Linux) 12 core / 32GB (dual processor) with 1TB RAID5 array running Ubuntu 16.04 (need to assemble hardware and install Linux) I would like to get the VMs up and running first; Will the Cloudera sandbox VM allow me to create a multi-node cluster with both virtual and hardware nodes?

Online	Offline
Last Visited	‎08-24-2017 05:26 PM

Date Registered	‎08-17-2017 02:29 PM
Last Visited	‎08-24-2017 05:26 PM
Total Messages Posted	6

Re: Hadoop cluster on my home network

Re: INTRODUCE YOURSELF: Please Reply here to tell ...

Re: Hadoop cluster on my home network

Re: Hadoop cluster on my home network

Hadoop cluster on my home network

Learning Hadoop; can I set up a training cluster a...