Member since
12-03-2016
5
Posts
0
Kudos Received
0
Solutions
03-28-2017
01:46 PM
Precise and clear! Thank you very much for your clarifications aarman! Regards, Luca
... View more
03-25-2017
11:23 AM
Hello, I have some doubts about a deployment of CDH on AWS. I read the reference architecture doc and other material I found on Cloudera Engineering Blog but I need some more insights about it. 1) Is the CDH deployment available only for some kind of instaces or I can deploy it on all the AWS instance types? 2) Assuming I want to create a cluster that will be active 24x7. For a long-running cluster I understood it's better to have a cluster based on local-storage instances. If we consider a cluster of 2PBs I think that d2.8xlarge should be the best choice for the datanodes. About the Master Nodes: - if I want to deploy only 3 Master Nodes, is it better to have them as local-storage instances too or as EBS attached instances to be able to react quickly to a possible Master Node failure? - are there some best practice about the master node instance type (EBS or local-storage)? About the Data Nodes: - if a data node fails, Has the CDH some automated mechanism to automaticly spin-up a new instance and connect it to the cluster in order to restore the cluster without down-times? Have we to build a script from scratch to do this thing? About the Edge Nodes: - are there some best practice about the instance type (EBS or local-storage)? 3) If I want to do a backup of the cluster on S3: - when I do a distcp from the CDH to S3, can I move the data directly on Glacier instead of the normal S3? If I have some compression applied on the data (e.g. snappy, gzip, etc.) and I do a distcp to S3: - Is the space occupied on S3 the same or the distcp command decompress the data for the copy? If I have a cluster based on EBS attached instances: - is it possible to snapshot the disks and re-attach a datanode based on the snapshot? 4) If I have the Data Nodes deploy as r4.8xlarge and I need more horsepower, is it possible to scale-up the cluster from r4.8xlarge to a r4.16xlarge on-the-fly? Attaching and detaching the disks in few mins? Thanks a lot for the clarifications, I hope my doubts will help also other users. 🙂 Best Regards, Luca
... View more
Labels:
- Labels:
-
Cloudera Manager
-
Manual Installation