Created 12-26-2018 04:57 PM
Ambari is showing hbase regionservers in a 'Decommissioning' status. When the regionservers are started it goes to the green checkmark (Installed status), then a few seconds later it goes to decommissioning status. The regionservers are up and appear to be running ok.
./hbase zkcli, ls /hbase-unsecure/draining and nothing is showing
./hbase-unsecure/rs and the regionservers are showing
The ambari database hostcomponentstate table shows regionservers 'STARTED'
I don't see anything in the ambari-agent log yet.
The ambari rest api shows desireds_state : STARTED for regionservers
How is the component status being set in the ambari console and how do I resolve this?
Created 12-28-2018 10:24 AM
Hi @Jack Madden ,
Sorry for the delay in response. I was analyzing deep in code why is this happening and it seems like its a BUG in javascript that it shows service is decommissioning stage when the HBASE region server is already started stage.
This looks like a javascript bug to me,
we can easily workaround this by fixing in app.js
Go through this below steps and let me know if it worked for you
Steps
1) go to ambari-server and navigate to below folder and take a app.js backup
[root@asn1 ~]# cd /usr/lib/ambari-server/web/javascripts/ [root@asn1 javascripts]# ls -lh total 29M -rw-r--r--. 1 root root 8.6M Dec 17 09:33 app.js -rw-r--r--. 1 root root 2.5M Dec 7 18:32 vendor.js [root@asn1 javascripts]# cp app.js app.js_backup_original
2) edit the app.js file
[root@asn1 javascripts]# vi app.js
this.setStatusAs('RS_DECOMMISSIONED');
and comment it
//this.setStatusAs('RS_DECOMMISSIONED');
3) save the file and Hard reload the ambari-web browser(open in incognito tab to remove the possbility of cached app.js)
Please share the feedback if it worked or not.
Please login and accept the answer if it did
Note : The following changes are based on my inspection and analysis. this BUG is not yet fixed in ambari, you can revert back the app.js after the change incases you wish for valid fix. i will raise apache bug for this and fix later.
Created 12-28-2018 10:26 PM
Thanks for digging into this and giving temporary workaround. I changed app.js and the regionserver shows started (green checkmark). The actions available list on the regionserver now are stop, turn on maintenance mode, and delete is disabled.
} else if (isInServiceDesired) { //this.setStatusAs('RS_DECOMMISSIONED');
this.setSatusAs('INSERVICE'); sets the regionserver to the correct status and correct actions list.
} else if (isInServiceDesired) { //this.setStatusAs('RS_DECOMMISSIONED'); this.setStatusAs('INSERVICE'); }
Thanks again for looking at this. How would I track the apache bug submission.
Created 12-29-2018 03:40 AM
Hey Jack,
Very much glad that it worked.
I have created an apache jira on this regard : https://issues.apache.org/jira/browse/AMBARI-25075
you can be yourself a jira watcher to this JIRA.