question Slow ambari rest api in Support Questions

Slow ambari rest api

ururu — Sat, 17 Aug 2019 23:48:38 GMT

Hello,

Can anyone explain how to rise speed of "rest api"?
Last month, the CPU load began to grow on ambari server and appeared alert "Ambari server performance rest api Critical 25,000ms"

I have 4 CPU core and 32GB ram on ambari host

The cluster is contain 25 nodes

Re: Slow ambari rest api

jsensharma — Tue, 12 Mar 2019 16:02:27 GMT

@Ruslan Fialkovsky

The following HCC article explains some of the points to check for Cluster API slow response scenario. Can you please check if that is helpful for you to troubleshoot the issue?

Section: Ambari API Response Time Check

https://community.hortonworks.com/articles/131670/ambari-server-performance-tuning-troubleshooting-c.html

Re: Slow ambari rest api

ururu — Tue, 12 Mar 2019 16:33:49 GMT

@Jay Kumar SenSharma

Thank you for your answer.

I have this "time requests"

time curl -i -u admin:* -H 'X-Requested-By: ambari' -X GET http://localhost:8080/api/v1/clusters/DataLake

real 0m36.270s
user 0m0.020s
sys 0m0.184s

time curl -i -u admin:* -H 'X-Requested-By: ambari' -X GET http://localhost:8080/api/v1/clusters/DataLake?fields=Clusters/desired_configs

real 0m0.190s
user 0m0.003s
sys 0m0.013s

time curl -i -u admin:* -H 'X-Requested-By: ambari' -X GET http://localhost:8080/api/v1/clusters/DataLake?fields=Clusters/health_report,Clusters/total_hosts,alerts_summary_hosts
real 0m30.498s
user 0m0.002s
sys 0m0.011s

I think something with database time response, isn't it?

Re: Slow ambari rest api

jsensharma — Tue, 12 Mar 2019 18:07:07 GMT

@Ruslan Fialkovsky

What is the age of your ambari managed cluster (like 6 monthds old / 1 year old / 2 year old) ?

After months of operation on larger clusters, the Ambari Server may begin to accrue a large amount of historical data in the database. This can cause UI performance degradation.

In some old clusters we see that there are lots of old "alert_history" (old operational logs) or old alert notification data entries present in the database that causes slowness, As with time these entries grows much on the database. So the DB dump size also grows and the DB queries can respond slow results. We can use the following command to perform some DB cleanup.

# ambari-server db-purge-history --cluster-name DataLake --from-date 2018-12-15

Reference: https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-administration/content/purging-ambari-server-history.html

Re: Slow ambari rest api

ururu — Tue, 12 Mar 2019 19:03:11 GMT

@Jay Kumar SenSharma

I already cleaned ambari database --from-date 2019-01-01 now it size is 1200mb, but i have databases "oozie, ranger, hive, airflow" on the same postgres cluster. Should i clean historical data from those databases?
HDP cluster is two years old.
The most big base now is oozie 12GB

Re: Slow ambari rest api

ururu — Sat, 17 Aug 2019 23:48:20 GMT

I have database replica and physical replication slot. When replica was down, postgers began to save wal journal and them was save all month! When i drop replication slot and clear old wal journal the cpu utilisation return to normal.