Cloudera API doesn't return any roles for a host

[#1]

Hi all,

I am starting working with the Cloudera API using python. My code is very simple:

# Print all hostnames and hostIds
for host in api.get_all_hosts():
print host.hostname + " - " + host.hostId
print host.roleRefs

I get values for the hostname and the hostId but NOTHING for roles.

I am using AWS with AutoScaling, so my idea is, when one datanode dies, create a Lambda to add
this new host to the cluster and assign the proper roles, so I want to check which hosts have
role already assigned or not.

My output :

All roles defined:
[u'DATANODE', u'NAMENODE', u'SECONDARYNAMENODE', u'BALANCER', u'GATEWAY', u'HTTPFS', u'FAILOVERCONTROLLER', u'JOURNALNODE', u'NFSGATEWAY']
[<cm_api.endpoints.types.ApiRoleRef object at 0x10d2dd0d0>, <cm_api.endpoints.types.ApiRoleRef object at 0x10d2dd6d0>]
-hadoopdatanode01.domain.com - f17671f2-b5a5-46c5-818e-5d560646fc52
None
-hadoopdatanode02.domain.com - cf2b59bc-63eb-4ca0-a917-fd6c6d9a4968
None
-hadoopdatanode03.domain.com - ccecb57c-d729-4177-a324-c2ec2655664f
None
-hadoopdatanode04.domain.com - ba7435a5-c406-43e8-94a0-bb112f8d7caa
None
-hadoopdatanode05.domain.com - f39cd9f8-f229-46bd-a187-b0f5e9121cc8
None

I have not idea what the problem could be. Roles have been assigned manually to these hosts using ClouderaManager

Regards

[#2 · Accepted solution]

Hi,

api.get_all_hosts() returns just basic information about a host by default, a SUMMARY view. You probably want the FULL view. See docs here:

https://cloudera.github.io/cm_api/epydoc/5.11.0/cm_api.api_client.ApiResource-class.html#get_all_hos...

If you do a simple HTTP GET on {CM_HOST:PORT}/api/v{version}/hosts you can easily see the kind of stuff returned by api.get_all_hosts(), and if you look at {CM_HOST:PORT}/api/v{version}/hosts?view=FULL you'll see you can get more details, like the role refs.

For your use-case of replacing a failed node, there's significant trickiness in getting the steps just right. You may want to look into Cloudera Director, which can repair worker or gateway nodes, among many other features. Here's the doc page for repairing a node:

https://www.cloudera.com/documentation/director/latest/topics/director_ui_cluster_shrink.html

Thanks,

Darren