How to remove roles on a given host ID on cludera API v11?

[#1]

I am trying to delete a host using cloudera API through the following:

API.get_cloudera_manager().decommission(hosts)
API.delete_hosts(hostID)

But it failed with error:

File "/usr/lib/python2.6/site-packages/cm_api/api_client.py", line 160, in delete_host
return hosts.delete_host(self, host_id)
File "/usr/lib/python2.6/site-packages/cm_api/endpoints/hosts.py", line 63, in delete_host
return call(resource_root.delete, "%s/%s" % (HOSTS_PATH, host_id), ApiHost)
File "/usr/lib/python2.6/site-packages/cm_api/endpoints/types.py", line 139, in call
ret = method(path, params=params)
File "/usr/lib/python2.6/site-packages/cm_api/resource.py", line 134, in delete
return self.invoke("DELETE", relpath, params)
File "/usr/lib/python2.6/site-packages/cm_api/resource.py", line 73, in invoke
headers=headers)
File "/usr/lib/python2.6/site-packages/cm_api/http_client.py", line 174, in execute
raise self._exc_class(ex)
cm_api.api_client.ApiException: Cannot delete host with associated roles. (error 400)

I know the problem was that the roles on the hosts should be deleted first. But I couldn't find a way how to do that. Does anyone have the corresponding function call?

Thanks!

[#2 · Accepted solution]

info can be found here:

https://cloudera.github.io/cm_api/epydoc/5.8.0/cm_api.endpoints.services.ApiService-class.html#delet...

-Ben

[#3]

That's it. Thanks so much bgooley!

[#4]

hi, 

can you explain how you solved it? 

im looking for a solution to remove all roles on a host- but not remove the role itself.. 

is that what you implement? if so- can you explain it? 

thanks,

-Maya

[#5]

Hello,

This python script helps to remove the hosts from the cluster. The following are the steps:

1. stop and decommission all roles in a host
2. remove the roles from a host
identify and delete the roles one by one
3. remove host from a cluster
4. remove host from cloudera manager

This script removes the hosts from the cloudera managed cluster running in aws. It is intend to scale down the worker node(node manager role) and gateway role from the cluster once the demand is over.

You can change the script accordingly based on your environment.

#!/bin/python
import httplib2
import os
import requests
import json
import boto3
import time
from requests.auth import HTTPBasicAuth
os.environ["AWS_ACCESS_KEY_ID"] = "ACCESS_KEY"
os.environ["AWS_SECRET_ACCESS_KEY"] = "SECRET_ACCESS_KEY"
os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
region='us-east-1'
metadata = requests.get(url='http://169.254.169.254/latest/meta-data/instance-id')
instance_id = metadata.text
host = requests.get(url='http://169.254.169.254/latest/meta-data/hostname')
host_id = host.text
username='admin'
password='admin'
cluster_name='cluster001'
scm_protocol='http'
scm_host='host.compute-1.amazonaws.com'
scm_port='7180'
scm_api='v17'
 
client = boto3.client('autoscaling')
ec2 = boto3.client('autoscaling', region_name=region)
response = client.describe_auto_scaling_instances(InstanceIds=[instance_id,])
state = response['AutoScalingInstances'][0]['LifecycleState']
print "vm is in " + state 
if state == 'Terminating:Wait':
print "host decommision started"
##decommission host
service_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/cm/commands/hostsDecommission'

#service_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/cm/hostsRecommission'
#service_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/cm/commands/hostsStartRoles'

print service_url
headers = {'content-type': 'application/json'}
req_body = { "items":[ host_id ]}
print req_body
req = requests.post(url=service_url, auth=HTTPBasicAuth(username, password), data=json.dumps(req_body), headers=headers)
print req.text
time.sleep(120)
##delete roles in a host
api_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/hosts/' + host_id
req = requests.get(api_url, auth=HTTPBasicAuth(username, password))
a = json.loads(req.content)

for i in a['roleRefs']:
scm_uri='/api/' + scm_api + '/clusters/' + cluster_name + '/services/'+i['serviceName']+'/roles/'+i['roleName']
scm_url = scm_protocol + '://' + scm_host + ':' + scm_port + scm_uri
print scm_url
req = requests.delete(scm_url, auth=HTTPBasicAuth(username, password))
print req.text
time.sleep(10)

##remove host from cluster
service_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/clusters/' + cluster_name + '/hosts/' + host_id
print service_url
req = requests.delete(service_url, auth=HTTPBasicAuth(username, password))
time.sleep(10)

##remove host from cloudera manager
os.system("/etc/init.d/cloudera-scm-agent stop") 
service_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/hosts/' + host_id
print service_url
req = requests.delete(service_url, auth=HTTPBasicAuth(username, password))
print req.text
time.sleep(10)
##refresh cluster configuration
service_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/clusters/' + 'commands/refresh'
print service_url
req = requests.post(service_url, auth=HTTPBasicAuth(username, password))
print req.text
time.sleep(10)
##deploy client configuration
service_url = scm_protocol + '://' + scm_host + ':' + scm_port + '/api/' + scm_api + '/clusters/' + 'commands/deployClientConfig'
print service_url
req = requests.post(service_url, auth=HTTPBasicAuth(username, password))
print req.text
time.sleep(10)

Best Regards,

Radhakrishnan Rk