Using api to configure datanode directories

[#1]

Hi.

A common administration task we have to perform is to add/remove entries from datanode and namenode configurations as hard disks break and are replaced. We would like to automate this task using the api if possible. 

I have had a look at https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/swagger-html-sdk-docs/python/README.html but cannot seem to find the endpoint that would allow me to configure DataNode Data Directory / dfs.datanode.data.dir and 

NodeManager Local Directories / yarn.nodemanager.local-dirs

Could someone let me know if this should be possible?

Cheers

[#2]

Hey @spotxsysops 

Thank you for bringing this to our community.

You may want to test this internally with this PUT call:

---

to configure DataNode Data Directory / dfs.datanode.data.dir and 

NodeManager Local Directories / yarn.nodemanager.local-dirs

Q1. To configure DataNode Data Directory / dfs.datanode.data.dir 

PUT /api/v31/clusters/Cluster%201/services/HDFS-1/roleConfigGroups/HDFS-1-DATANODE-BASE/config

{ "items": [ { "name": "dfs_data_dir_list", "value": "/path/to/dir" } ] }

Q2. To configure NodeManager Local Directories / yarn.nodemanager.local-dirs

PUT /api/v31/clusters/Cluster%201/services/YARN-1/roleConfigGroups/YARN-1-NODEMANAGER-BASE/config

{ "items": [ { "name": "yarn_nodemanager_local_dirs", "value": "/path/to/dir" } ] }

However, The Generic CM API End-Points for RoleConfigGroups can be found here:

https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/resource_RoleConfigGroupsResource....

Let me know if this helps?

[#3]

Hi vaishaakb.

Thank you for your response, that is very helpful.

Could I impose on you further and and ask how I would do it to individual hosts / roleids? 

Thank you again.

-- 

Pete

[#4]

Hey Pete!

I appreciate your patience! Sorry, I was away.

> Could I impose on you further and and ask how I would do it to individual hosts / roleids? 

Sure, I will be happy to help you with that.

> how I would do it to individual hosts / roleids? 

For this, You would want to get the hostids from CM-API

Example:
$ curl -u <admin-user>:<admin-password> http://<CM_SERVER_FQDN>:7180/api/v44/hosts

Followed by the above, You can try entering replacing the hostId in the items section[0]

[0] https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/json_ApiRoleList.html

Let me know if this helps?

V

[#5]

Hi Vaishakkb.

Thank you again for your response. 

Unfortunately I am unable to make it work. Looking at the docs (https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/resource_RoleConfigGroupsResource....) for that endpoint, it seems to want an ApiConfigList rather than an ApiRoleList. If I try to use an ApiRoleList object I get "message" : "Unrecognized property: for anything I put in it after "items".

Cheers

-- 

Pete

[#6]

@spotxsysops, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.