Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Using api to configure datanode directories

avatar
Explorer

Hi.

 

A common administration task we have to perform is to add/remove entries from datanode and namenode configurations as hard disks break and are replaced. We would like to automate this task using the api if possible. 

 

I have had a look at https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/swagger-html-sdk-docs/python/README.html but cannot seem to find the endpoint that would allow me to configure DataNode Data Directory / dfs.datanode.data.dir and 

NodeManager Local Directories / yarn.nodemanager.local-dirs
 
Could someone let me know if this should be possible?
 
Cheers
 
 
5 REPLIES 5

avatar
Master Collaborator

Hey @spotxsysops 

Thank you for bringing this to our community.

 

You may want to test this internally with this PUT call:


to configure DataNode Data Directory / dfs.datanode.data.dir and 

NodeManager Local Directories / yarn.nodemanager.local-dirs
Q1. To configure DataNode Data Directory / dfs.datanode.data.dir 
 
PUT /api/v31/clusters/Cluster%201/services/HDFS-1/roleConfigGroups/HDFS-1-DATANODE-BASE/config

{ "items": [ { "name": "dfs_data_dir_list", "value": "/path/to/dir" } ] }
 
Q2. To configure NodeManager Local Directories / yarn.nodemanager.local-dirs
 
PUT /api/v31/clusters/Cluster%201/services/YARN-1/roleConfigGroups/YARN-1-NODEMANAGER-BASE/config

{ "items": [ { "name": "yarn_nodemanager_local_dirs", "value": "/path/to/dir" } ] }
 
However, The Generic CM API End-Points for RoleConfigGroups can be found here:
 
Let me know if this helps?

avatar
Explorer

Hi vaishaakb.

 

Thank you for your response, that is very helpful.

 

Could I impose on you further and and ask how I would do it to individual hosts / roleids? 

 

Thank you again.

 

-- 

Pete

avatar
Master Collaborator

Hey Pete!

 

I appreciate your patience! Sorry, I was away.

 

> Could I impose on you further and and ask how I would do it to individual hosts / roleids? 

Sure, I will be happy to help you with that.

 

how I would do it to individual hosts / roleids? 

For this, You would want to get the hostids from CM-API

Example:
$ curl -u <admin-user>:<admin-password> http://<CM_SERVER_FQDN>:7180/api/v44/hosts

 

Followed by the above, You can try entering replacing the hostId in the items section[0]

[0] https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/json_ApiRoleList.html

Let me know if this helps?

 

V

avatar
Explorer

Hi Vaishakkb.

 

Thank you again for your response. 

 

Unfortunately I am unable to make it work. Looking at the docs (https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/resource_RoleConfigGroupsResource....) for that endpoint, it seems to want an ApiConfigList rather than an ApiRoleList. If I try to use an ApiRoleList object I get "message" : "Unrecognized property: for anything I put in it after "items".

 

Cheers

 

-- 

Pete

 

avatar
Community Manager

@spotxsysops, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: