Reply
Explorer
Posts: 7
Registered: ‎04-10-2015
Accepted Solution

Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

Custom add on service stuck in starting state and now cannot start,stop or delete the service itself. 

When I try to start I get the error "Command Start is not currently available for execution."

Whey I try to stop the service I get the error "At least one role must be started."

When I try to delete the service I get the error "The following roles roles need to be stopped before deleted."

 

Please advise.

Posts: 947
Kudos: 30
Solutions: 18
Registered: ‎05-27-2014

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

Follow Cloudera Knowledge Article: "Cloudera Manager managed role stuck in stopping or starting state" resolved the issue.

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

New Contributor
Posts: 2
Registered: ‎04-07-2016

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

Looks like I have the same issue, can you post the url of the article in question please?
Posts: 947
Kudos: 30
Solutions: 18
Registered: ‎05-27-2014

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

Hi,

 

I am copying the solution here in case you don't have access to the KB article.

 

Please make sure you have a Cloudera Manager database backup.

 

Perform the following actions:
1. Log into the Cloudera Manager database and run a query against all the services in the Roles table.

select role_id, name, configured_status from ROLES where configured_status = "STOPPING"; -- Replace STOPPED with STARTING depending on the use case.

2. Note any roles that you know for certain are stopped but are indicated as "STOPPING" or any roles you know for certain are started but are indicated as "STARTING"
3. Execute an update statement to set the role to the correct state.

update ROLES set configured_status = "STOPPED" where role_id = #; -- Replace STOPPED with STARTING depending on the use case. Replace the role_id with the value got from step 1.
4. Re-execute the select statement to check to ensure the configured_status is now indicating the correct state.

5. Start or stop the role as required.

 

Hope this helps.

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

New Contributor
Posts: 2
Registered: ‎04-07-2016

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

[ Edited ]

This is exactly what I needed. Many thanks. Solved.

New Contributor
Posts: 4
Registered: ‎04-05-2016

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

It should restart cloudera-scm-server services after updated the DB.

Explorer
Posts: 18
Registered: ‎01-29-2018

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

Hi @lwang,

 

Thanks a lot to share this information with us! This is an odd behavior, could you tell me what is the root cause of this, please? Wich CDH version are impacted? At the moment I have found this problem with CDH 5.8 only.

 

Many thanks in advance for the kind cooperation/availability.

 

Regards,

 

Alex

Posts: 947
Kudos: 30
Solutions: 18
Registered: ‎05-27-2014

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

[ Edited ]

Hi @AlexPQ,

 

The issue was mostly seen during cluster upgrade. Rarely it can oocur in below scenarios:

- backend CM database connectivity going down during command execution

- agent stopping while the command was running

- some unknown reasons

 

We have some internal jiras associated with the issue. Based on the discussions there, it looks like there were a couple of bugs involved state transition. Both bugs were resolved in CM5.12.0 and higher. This may explain why in your cluster (assuming you have CM 5.8.x as same as your CDH version) you are seeing this issue.

 

Thanks and hope this helps.

 

 

 

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

New Contributor
Posts: 1
Registered: ‎10-18-2018

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

[ Edited ]

.

 

Explorer
Posts: 18
Registered: ‎01-29-2018

Re: Custom add on service stuck in starting state, and now cannot start,stop,delete this service.

Hi @lwang


thank you very much for your kind feedback, I'm preparing the CDH and CM upgrade to 5.13.1, in order to avoid this issue.

 

Thanks again!

 

Regards,

 

Alex

Announcements