Reply
New Contributor
Posts: 1
Registered: ‎11-16-2018

Install Cloudera Cluster on AWS? x/post from r/hadoop

I'm trying to create a 4 node Hadoop cluster on AWS using the Cloudera distribution for learning purposes.

I've done it manually (without Cloudera) and ended up spending more time just learning about Linux/Networking/Matching correct software versions etc. and less time using Hadoop/Map Reduce/Hive, which is what I really want to focus on.

I'd much rather learn these Hadoop technologies in a multi node cluster environment instead of a single node VM. I feel like I'm not really learning everything when it's just a single node. I also want to work with the Cloudera distro in particular and use HUE.

I've found videos/articles from a few years ago that reference being able to set up a Cloudera Live account and run it on AWS, however I can't find anything like that on Cloudera's website anymore.

Does anyone have advice? Can this be done relatively simply and low cost? Are there better ways to practice on a multi node cluster using Cloudera?

Thanks

Highlighted
Cloudera Employee
Posts: 207
Registered: ‎01-15-2015

Re: Install Cloudera Cluster on AWS? x/post from r/hadoop

One option you may have missed would be the Cloudera QuickStart images to play with, see https://www.cloudera.com/downloads/quickstart_vms/5-13.html

But this is for single instance testing only.

For installing a multiple node test cluster in the cloud the following Cloudera blog articles may help you:

http://blog.cloudera.com/blog/2016/05/how-to-deploy-a-secure-enterprise-data-hub-on-aws/
http://blog.cloudera.com/blog/2017/01/how-to-deploy-a-secure-enterprise-data-hub-on-microsoft-azure-...
http://blog.cloudera.com/blog/2018/11/learn-for-free-how-to-deploy-cloudera-enterprise-on-microsoft-...

You may also want to try the Cloudera Altus product 30 day trial, see https://www.cloudera.com/products/altus.html
Announcements