cloudera director fonctionalities

[#1]

Hello,

We wanted to deploy a cluster with Cloudera Director with 2 master nodes and 4 slaves nodes. 

However, we are having some problems:

- we cannot add more than 1 master in cloudera director

- we cannot make the cluster without a gateway node

- we cannot choose a custom service package (we wanted a package with Impala+HBase but without spark/solr)

- we cannot change the admin password/make another user

Please, can you indicate a documentation that can help us with this ? 

Is there any feature list documentation for cloudera director?

What are the benefits of using Cloudera Director insteaed  of Cloduera Manager for deployment?

Thank you!

[#2 · Accepted solution]

Hi Alina!

We're working on exposing this functionality via the user interface at present.  This functionality does exist within Director at present, though currently less than optimal.

First, I'll pick the easy to answer question.  For documentation as to features as well as cluster setup instructions, you can refer to the documentation here:http://www.cloudera.com/content/cloudera/en/documentation/cloudera-director/latest/PDF/cloudera-dire....

For user management, including addition of users and changing the admin password, you'll need to access the API console.  You can do this by clicking on the question mark at the top right of the screen and selecting "API console" from the drop down list.  This can also be accessed by navigating to the /api-console/ endpoint of your Cloudera Director server.

 

From here, you'll have access to all of the user management functionality currently within Director.  All of the relevant operations will be under the "users" group of operations.

For adding a new user, you'll want to look at the POST /api/v1/users endpoint.  You'll need to create a JSON like structure to describe the new user being created.  As an example:

{

    "enabled": true,

    "password": "a-password",

    "roles": ["ROLE_ADMIN", "ROLE_READONLY"],

    "username": "test-user"

}

When you're ready to create a user, click "Try it out" at the bottom of the window.  On success, the response code should be "201."

Note: There are only two roles that are supported at present: ROLE_ADMIN and ROLE_READONLY.  READONLY will only allow read access to Director, while ADMIN will allow cluster creation.

For updating the admin password, you'll want to look at the /api/v1/users/{username}/password endpoint.  You'll need to create a JSON structure to describe the password change.  As an example:

{

    "oldPassword": "admin",

    "newPassword": "a-new-test-password"

}

Like creating a user, you'll need to click "Try it out!" at the bottom of the page to change the password.  A response code of "202" here indicates success.

​

In regards to more flexible cluster creation: at present, this can only be done via the command line interface (CLI), which can be downloaded and installed via our repositories.  The package name is cloudera-director-client.  This should be used in conjunction with the Cloudera Director server.

Once you've installed this, you should be able to find a README and relevant sample files within /usr/lib64/cloudera-director/client.  Of particular interest will be aws.reference.conf, which lists and documents all of the functionality that the Cloudera Director client provides.  Once you've configured your conf file as expected, you'll want to use the bootstrap-remote command like this:

cloudera-director custom-deployment.conf --lp.remote.hostAndPort=director-host:7189 --lp.remote.username=admin

This should prompt you for a password prior to bootstrapping your deployments and clusters.

As far as using Director vs. Manager for deployment, they're really intended as complementary services.  Director can eliminate the need to provision and configure instances, which would be necessary when using Cloudera Manager on its own.

[#3 · Accepted solution]

Hi Alina!

We're working on exposing this functionality via the user interface at present.  This functionality does exist within Director at present, though currently less than optimal.

First, I'll pick the easy to answer question.  For documentation as to features as well as cluster setup instructions, you can refer to the documentation here:http://www.cloudera.com/content/cloudera/en/documentation/cloudera-director/latest/PDF/cloudera-dire....

For user management, including addition of users and changing the admin password, you'll need to access the API console.  You can do this by clicking on the question mark at the top right of the screen and selecting "API console" from the drop down list.  This can also be accessed by navigating to the /api-console/ endpoint of your Cloudera Director server.

 

From here, you'll have access to all of the user management functionality currently within Director.  All of the relevant operations will be under the "users" group of operations.

For adding a new user, you'll want to look at the POST /api/v1/users endpoint.  You'll need to create a JSON like structure to describe the new user being created.  As an example:

{

    "enabled": true,

    "password": "a-password",

    "roles": ["ROLE_ADMIN", "ROLE_READONLY"],

    "username": "test-user"

}

When you're ready to create a user, click "Try it out" at the bottom of the window.  On success, the response code should be "201."

Note: There are only two roles that are supported at present: ROLE_ADMIN and ROLE_READONLY.  READONLY will only allow read access to Director, while ADMIN will allow cluster creation.

For updating the admin password, you'll want to look at the /api/v1/users/{username}/password endpoint.  You'll need to create a JSON structure to describe the password change.  As an example:

{

    "oldPassword": "admin",

    "newPassword": "a-new-test-password"

}

Like creating a user, you'll need to click "Try it out!" at the bottom of the page to change the password.  A response code of "202" here indicates success.

​

In regards to more flexible cluster creation: at present, this can only be done via the command line interface (CLI), which can be downloaded and installed via our repositories.  The package name is cloudera-director-client.  This should be used in conjunction with the Cloudera Director server.

Once you've installed this, you should be able to find a README and relevant sample files within /usr/lib64/cloudera-director/client.  Of particular interest will be aws.reference.conf, which lists and documents all of the functionality that the Cloudera Director client provides.  Once you've configured your conf file as expected, you'll want to use the bootstrap-remote command like this:

cloudera-director custom-deployment.conf --lp.remote.hostAndPort=director-host:7189 --lp.remote.username=admin

This should prompt you for a password prior to bootstrapping your deployments and clusters.

As far as using Director vs. Manager for deployment, they're really intended as complementary services.  Director can eliminate the need to provision and configure instances, which would be necessary when using Cloudera Manager on its own.

[#4 · Accepted solution]

Hey Alina.  After discussing this with my colleagues, I wanted to add that there's another option you have for flexible cluster creation.  You can utilize the API console.  Look under clusters for the POST /api/v1/environments/{environment}/deployments/{deployment}/clusters endpoint:

This will allow you to create a JSON structure that describes a cluster in the way that you desire.  It's not as well documented and will require a bit more experimentation to create a cluster correctly.  You can get an example JSON by creating a cluster and then navigating to to the /api/v1/environments/{environment}/deployments/{deployment}/clusters/{cluster}/template endpoint.  For example, I made an environment called "test-env", a deployment called "test-cm", and a cluster called "test-cluster".  By navigating to:

/api/v1/environments/test-env/deployments/test-cm/clusters/test-cluster/template

I get the following output:

Which I can then use as a reference to build a template out of.

You can use either method, the API console or the command line interface, but both will accomplish what you'd like.  I just wanted to make sure you had both options!