How to remove all previous version HDP directories from /usr/hdp using Ambari.

[#1]

Post HDP upgrade you will observe all previous version HDP directories will be exist under /usr/hdp/ directory (Including current active). Once upgrade is finalized you can clear those older version directories using Ambari. use following "curl" command.

1. Ssh to ambari server and run following command. (You can use any linux box in network which has curl installed and able to access ambari console URL).

curl 'http://<$AMBARI_SERVER>:8080/api/v1/clusters/<$MyClusterName>/requests' -u admin:<Password> -H "X-Requested-By: ambari" -X POST -d'{"RequestInfo":{"context":"remove_previous_stacks", "action" : "remove_previous_stacks", "parameters" : {"version":"<$Current_Active_version>"}}, "Requests/resource_filters": [{"hosts":"<$ClusterNode1>, <$ClusterNode2>"}]}'

• $MBARI_SERVER => Ambari server name and port. If you are using SSL use the SSL port no.
• $MyClusterName => Enter cluster name you want to remove directories.
• $ Current_Active_version => Enter current active version. (It will remove all previous versions directory.) It will remove all older versions of mentioned current.
• $ClusterNode1 => Enter your cluster node HostName/IP. (Using hostname is recommended)

for e.g

curl 'http://myambariserver.test.com:8080/api/v1/clusters/myprodcluster1/requests' -u admin:xxxxxx -H "X-Requested-By: ambari" -X POST -d'{"RequestInfo":{"context":"remove_previous_stacks", "action" : "remove_previous_stacks", "parameters" : {"version":"2.6.1.0-129"}}, "Requests/resource_filters": [{"hosts":"datanode1.test.com, datanode2.test.com,mstr1.test.com"}]}'

Output:

{

"href" : "http:// myambariserver.test.com:8080/api/v1/clusters/myprodcluster1/requests/830",

"Requests" : {

"id" : 830,

"status" : "Accepted"

}

}

2. Track the progress for each cluster nodes from running operations.

Login to Ambari console=>click on ops (Right corner near cluster name). You will see running operation "remove_previous_stacks".

NOTE: If you have long list of cluster nodes, you can specify list of nodes in one file and provide full path of file to the host directive.

for e.g.

[{"hosts":"/var/tmp/clusternodes1.txt"}]}'

[#2]

The option to use a file that contains the node FQDNs does not seem to work. The command thinks that the filename is a node FQDN. Also, what is the format of the clusternodes1.txt? One host per line or all hosts comma separated on one line?

[#3]

I've tried this procedure on a freshly upgraded HDP setup (version 3.0.1). But when I launch the request it immediately errors out with this in the output:

stdout:

	2018-12-19 10:38:04,388 - call[('ambari-python-wrap', u'/usr/bin/hdp-select', 'versions')] {}
	2018-12-19 10:38:04,416 - call returned (0, '2.6.2.14-5\n2.6.5.0-292\n3.0.1.0-187')
	2018-12-19 10:38:04,416 - available versions: ['2.6.2.14-5', '2.6.5.0-292', '3.0.1.0-187']
	2018-12-19 10:38:04,417 - version 2.6.2.14-5 added to remove

	Command failed after 1 tries

stderr:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 119, in <module>
  RemovePreviousStacks().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 351, in execute
  method(env)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 49, in actionexecute
  self.remove_stack_version(structured_output, low_version)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 54, in remove_stack_version
  packages_to_remove = self.get_packages_to_remove(version)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 77, in get_packages_to_remove
  all_installed_packages = self.pkg_provider.all_installed_packages()
AttributeError: 'YumManager' object has no attribute 'all_installed_packages'

A quick google did not return anything usefull. But it looks like the some of the supporting code for this is missing or we are missing a extra library somewhere?

[#4]

@Tim Verhoeven,

It will be helpfull if you could kindly create a new thread for issue you are facing.

[#5]

The Abouve Article doesnt work in ambari-2.7.3 due to a bug .

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 119, in <module>
    RemovePreviousStacks().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 49, in actionexecute
    self.remove_stack_version(structured_output, low_version)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 54, in remove_stack_version
    packages_to_remove = self.get_packages_to_remove(version)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 77, in get_packages_to_remove
    all_installed_packages = self.pkg_provider.all_installed_packages()
AttributeError: 'YumManager' object has no attribute 'all_installed_packages'

Please refer to article if you face the same bug : https://community.hortonworks.com/articles/230893/remove-old-stack-versions-script-doesnt-work-in-am...