Reply
Expert Contributor
Posts: 66
Registered: ‎12-24-2015
Accepted Solution

root volume on ec2 instances ?

I have three questions on instance storage on ec2 instance.

 

1. How to give different volume for workers and master, if using director client ?

    rootVolumeSizeGB: <??>

 

2. is the above root volumne added is of EBS kind ?

 

3. For example, If i select worker machine of type d2.2xlarge. what is use of  OOTB instance storage attached to EC2 volume since that is never mounted for root expansion ?


d2.2xlarge ===> 6 x 2048 GB (Use ?)

Cloudera Employee
Posts: 11
Registered: ‎09-21-2015

Re: root volume on ec2 instances ?

Hi Kartikbha,

 

When defining a config, the instances section is basically a set of instance templates that you are defining.

So for your worker group, you could define a worker template, and your master group would use the master template.  Given this separation, you would assign the worker template to your worker cluster group definition.

 

instances {

  worker-instance {

    type: d2.2xlarge

    rootVolumeSizeGB: 100 # 100GB

    ...

  }

  master-instance {

    type: super.powered.xxxlarge

  }

}

 

....

 

cluster {

  masters {

    instance: ${master-instance}

  }

  workers {

   instance: ${worker-instance}

  }

}

 

 

I think that EBS root volume type is specific to the ami you choose.

Looking at the aws documentation:

https://aws.amazon.com/ec2/instance-types/

 

It looks like d2.2xlarge doesn't say anything about EBS, so I may be assuming incorrectly but I don't think it is of the EBS kind.

Cloudera Employee
Posts: 11
Registered: ‎09-21-2015

Re: root volume on ec2 instances ?

Re: #2, I just asked and the root volume size that you specify in the config file is for an EBS backed volume.

 

Expert Contributor
Posts: 66
Registered: ‎12-24-2015

Re: root volume on ec2 instances ?

Hi jheyming, question #3 is not answered.

Regards,
Highlighted
Cloudera Employee
Posts: 11
Registered: ‎09-21-2015

Re: root volume on ec2 instances ?

Hi kartikbha,

 

I asked what we do and we mount them as /data0, /data1, ... etc

 

It was mentioned that they are used for hdfs data.