HDP Downgrade Issue - Kafka service failed to stop

[#1]

ISSUE:

1. While performing HDP upgrade from 2.4.2 to 2.5.0 and which failed on last step Finalize upgrade saying few hosts are not able to upgrade to latest version.
2. Tried to revert the upgrade and proceeded with Downgrade option.
3. While Downgrade it prompt for Atlas and Kafka to be deleted from Cluster. 
4. Deleted Atlas and Kafka from cluster.
5. Proceeding further the stopping of service failed on to stop KAFKA
6. The downgrade screen was paused and not it was showing in Ambari UI details tab that it "Failed to start KAFKA_BROKER"

ROOT CAUSE:

Looks like there are two tasks stuck in the PENDING state from the downgrade

RESOLUTION:

1. You need to check at what step the upgrade is stuck using below command -

http://<ambari_host>:8080/api/v1/clusters/<clustername>/upgrades/

2. Pick the latest "request_id" from above output and execute -

http://<ambari_host>:8080/api/v1/clusters/<clustername>/upgrades/<request_id>;

In my case request_id was 858

3. Login to ambari database and use below command with "request_id" to check task which are not in COMPLETED status in host_role_command table as shown below -

ambari=> SELECT task_id, status, event, host_id, role, role_command, command_detail, custom_command_name FROM host_role_command WHERE request_id = 858 AND status != 'COMPLETED' ORDER BY task_id DESC

8964, PENDING, 4, KAFKA_BROKER, CUSTOM_COMMAND, RESTART KAFKA/KAFKA_BROKER, RESTART
8897, PENDING, 4, KAFKA_BROKER, CUSTOM_COMMAND, STOP KAFKA/KAFKA_BROKER, STOP

4. Update the status of the above task to COMPLETED using below command -

UPDATE host_role_command SET status = 'COMPLETED' WHERE request_id = 858 AND status = 'PENDING';

After which it was able to proceed with Downgrade.