Created 10-25-2018 04:36 AM
After fresh installation I got the following critical error alert in YARN:
Title: ATSv2 HBase Application
Response:
Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/alerts/alert_ats_hbase.py", line 183, in execute ats_hbase_app_info = make_valid_json(output) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/alerts/alert_ats_hbase.py", line 226, in make_valid_json raise Fail("Couldn't validate the received output for JSON parsing.") Fail: Couldn't validate the received output for JSON parsing.
Created 10-25-2018 05:39 AM
Can you please check if the following "status" command is giving you proper output?
# su - yarn-ats -c "/usr/hdp/current/hadoop-yarn-client/bin/yarn app -status ats-hbase"
If "ats-hbase" service is down the you might see the error. Because the failing alert will also run the same query and then will attempt to parse the JSON output so if the status is not returning fine they you should see the alert.
Please check if your "ats-hbase" service is running?
Reference:
https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/data-operating-system/content/options_to_re...
https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/data-operating-system/content/enable_system...
Created 10-25-2018 05:44 AM
Ambair also allows to disable alerts .. so if it is your test cluster and if you just want to suppress that alert then just click on that alert link in ambari UI and "Disable" button in the alerts page.
Created 10-26-2018 06:43 AM
@Jay Kumar SenSharma
Hi Jay Kumar!
The (# su - yarn-ats -c "/usr/hdp/current/hadoop-yarn-client/bin/yarn app -status ats-hbase") output :
[root@hadoop anaconda3]# su - yarn-ats -c "/usr/hdp/current/hadoop-yarn-client/bin/yarn app -status ats-hbase"
18/10/25 13:18:42 INFO client.RMProxy: Connecting to ResourceManager at hadoop.test.com/xxx.xxx.xxx.xxx:8050
18/10/25 13:18:42 INFO client.AHSProxy: Connecting to Application History server at hadoop.test.com/xxx.xxx.xxx.xxx:10200
18/10/25 13:18:42 INFO client.RMProxy: Connecting to ResourceManager at hadoop.test.com/xxx.xxx.xxx.xxx:8050
18/10/25 13:18:42 INFO client.AHSProxy: Connecting to Application History server at hadoop.test.com/xxx.xxx.xxx.xxx:10200
18/10/25 13:18:42 INFO util.log: Logging initialized @1294ms
ats-hbase Failed : HTTP error code : 500
[root@hadoop anaconda3]#
As I understand the service "ats-hbase" is down. To start this service I should follow the instructions given in - https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/data-operating-system/content/enable_system...
Am I right ? How important this service (ats-hbase) for production ?
Thank you!
Created on 07-29-2020 12:08 PM - edited 07-29-2020 12:16 PM
Here, <app-id> is the ID of the ats-hbase service.
I stopped at this step and the alert notification still showing although I have restart the yarn and refresh yarn capacity scheduler, how I can find the app-id?
I have try " yarn app -status ats-hbase " but it returns
20/07/30 01:42:49 INFO client.RMProxy: Connecting to ResourceManager at hdpdev02.bps.go.id/10.0.45.112:8050
20/07/30 01:42:49 INFO client.AHSProxy: Connecting to Application History server at hdpdev02.bps.go.id/10.0.45.112:10200
20/07/30 01:42:50 INFO client.RMProxy: Connecting to ResourceManager at hdpdev02.bps.go.id/10.0.45.112:8050
20/07/30 01:42:50 INFO client.AHSProxy: Connecting to Application History server at hdpdev02.bps.go.id/10.0.45.112:10200
ats-hbase Failed : HTTP error code : 500
NB: The cluster is a kerberized cluster
Created 05-30-2019 05:24 AM
Created 05-30-2019 08:07 AM
Hi, could you post here you alert message ?
Created 05-30-2019 11:10 AM
Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/alerts/alert_ats_hbase.py", line 183, in execute ats_hbase_app_info = make_valid_json(output) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/alerts/alert_ats_hbase.py", line 226, in make_valid_json raise Fail("Couldn't validate the received output for JSON parsing.") Fail: Couldn't validate the received output for JSON parsing.
Created 05-30-2019 03:06 PM
what version of OS, HDP do you use ?
Created 05-31-2019 06:50 AM