- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Cluster Scaling via Director API
Created on 01-25-2017 10:06 AM - edited 09-16-2022 08:19 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I understand that scaling the cluster is done by redefining the cluster itself. What I don't know (and have been unable to find) is how much information do I need to include in PUT to change the cluster. There's a lot of information that is generated automatically (IP addresses for example) that I don't have/want to include, I'd rather just say I want X nodes of Y template. Anyone have a resource I can look at? I've spent sometime working in the API console and the documentation just seems lacking here...
Created 01-27-2017 07:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So you'll really need PUT for the cluster template in order to make an update rather than the cluster object. The cluster object is a bit more ephemeral in nature and reflects the current active state of the cluster.
Are you using one of the SDKs (https://github.com/cloudera/director-sdk, only available in Java/Python)? It sounds like you're doing something a little more manual, but I'd encourage you to use those if possible. Otherwise, the API console is probably your best bet for documentation at the moment. Assuming Director is running on your local machine, you'd navigate to:
http://localhost:7189/api-console
Let me know if that points you in the right direction.
Created 01-27-2017 07:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So you'll really need PUT for the cluster template in order to make an update rather than the cluster object. The cluster object is a bit more ephemeral in nature and reflects the current active state of the cluster.
Are you using one of the SDKs (https://github.com/cloudera/director-sdk, only available in Java/Python)? It sounds like you're doing something a little more manual, but I'd encourage you to use those if possible. Otherwise, the API console is probably your best bet for documentation at the moment. Assuming Director is running on your local machine, you'd navigate to:
http://localhost:7189/api-console
Let me know if that points you in the right direction.
Created 01-27-2017 08:26 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My original thought was to talk JSON to the API directly. The API console is where I've been spending my time trying to understand what's needed to resize the cluster. I'd glanced at the sdk, but looking at it further, it might work for what I'm trying to do (faux EMR).
Created 01-30-2017 09:45 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 11-09-2017 12:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Mike Wilson : Hello ,
Can you please provide a high level over view i should follow for the below requirement.
Requirement: We want to develop a script which will perform instance addition, master service migration,datanode replacement.
On our cluster we have Director 2.6 and use AWS as infra.
Does cloudera director provide api's to perform the above service .I guess the above requirement will need a mix of director and manager api's.
Is ansible suitable for the above requirement.
Created 11-09-2017 01:34 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Tauqeer,
Cloudera Director's API will provide what you need to add instances and replace data nodes. Master migration is a little trickier. Cloudera Director can help with creating instances for the migration, but you will have to perform the migration in Cloudera Manager. See https://www.cloudera.com/documentation/director/latest/topics/director_create_ha_clusters.html#migra... for information on how to use Cloudera Director when migrating masters.
These tasks will mainly use Cloudera Director's "update" API. See https://github.com/cloudera/director-sdk/blob/master/java-client/src/main/java/com/cloudera/director... or https://github.com/cloudera/director-sdk/blob/master/python-client/cloudera/director/latest/Clusters...
There are examples in both java and python to get you started.
Created 11-09-2017 05:17 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
