Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

cloudera-director update aws.reference.conf - Not able to update workers using director client.

cloudera-director update aws.reference.conf - Not able to update workers using director client.

Explorer

we have upper limit for EC2 machines under d2.2x category but no such limit for i2.x EC2.

 

Here is what I did.

 

Started Cluster using director bootstrap command with just one worker of type d2.2x.

 

Now i am trying to update my cluster with 2 new worker of type i2.x EC2. I am still receving old error which i was getting with d2.2x.

 

com.cloudera.director.aws.shaded.com.amazonaws.AmazonServiceException: You have requested more instances (2) than your current instance limit of 1 allows for the specified instance type. Please visit http://aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit. (Service: AmazonEC2; Status Code: 400; Error Code: InstanceLimitExceeded; Request ID: c999a493-11e1-4b55-b07f-fdc717310943)
--------------------------------------------------

 

my conf is like below :

 

## exisitng worker

workers {
count: 1
minCount: 1

instance: ${instances.d22x} {

tags {
group: worker
}
}

roles {
HDFS: [DATANODE]
YARN: [NODEMANAGER]
HBASE: [REGIONSERVER]
}

}


#### my new workers
workers2 {
count: 3
minCount: 3

instance: ${instances.i2x} {

tags {
group: worker2
}
}

roles {
HDFS: [DATANODE]
YARN: [NODEMANAGER]
HBASE: [REGIONSERVER]
}

}

 

 

 

 

 

 

 

 

 

10 REPLIES 10

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Contributor

Hi,

 

 

Since your conf file contains the old worker configs, it may be trying to add that a second time. Can you try with just the new worker groups?

 

Alternately you can also add a new group from the UI, in which case it won't impact the existing group.

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Explorer

Hi Jayita,

 

Thanks for reply.

 

I have kept only new worker group and run update command. It did not work.

 

Are you saying me to add workers from cloudera manager UI, if you remember in this there is no director UI for me.

 

 

 

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Cloudera Employee

The director ui should be running on port 7189 of your director host.

 

I'm wondering if the conf file you are supplying has the old worker group present, so director doesn't think it needs to delete the old workers group.  I'm going to try out a bootstrap remote command myself and try to give a concrete example of removing instances from a group, or changing the instances completely

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Cloudera Employee

Ok, so, after testing this out, there is no way to use bootstrap-remote to modify a cluster.  You have to either use the UI or the API (python/java client) 

 

 

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Explorer
So u mean update command from director client can't add workers ?
Highlighted

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Cloudera Employee

Bootstrap remote is used mainly to get a cluster up and running.  Once its running, if you run a second bootstrap remote, it won't modify the existing cluster groups.  That includes adding/removing groups.

 

You need to use the api to modify the cluster

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Explorer
I didn't use "bootstrap remote" command to build cluster. I used simple "bootstrap" followed by "update" command.

Where to look for API

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Cloudera Employee

I see.  I just looked at the documentation and it looks like bootstrap-remote is used to get an existing Director server to create a cluster.  

http://www.cloudera.com/documentation/director/latest/topics/director_cli_commands.html

 

Bootstrap and update are used to make changes to an existing cloudera manager.

 

So we are talking about different things.  Can you confirm what your setup is?

Re: cloudera-director update aws.reference.conf - Not able to update workers using director client.

Explorer
I used only "bootstrap" command to Start cluster using director client.
Now trying to expand cluster using "update" command.
Don't have an account?
Coming from Hortonworks? Activate your account here