Member since
08-08-2017
1652
Posts
30
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2058 | 06-15-2020 05:23 AM | |
| 17011 | 01-30-2020 08:04 PM | |
| 2220 | 07-07-2019 09:06 PM | |
| 8552 | 01-27-2018 10:17 PM | |
| 4835 | 12-31-2017 10:12 PM |
02-20-2020
12:18 PM
hi all
we have HDP version 2.6.5 and ambari 2.6.2.2
we configured the following
Metrics Service operation mode=distributed
hbase.rootdir=hdfs://hdfsha:8020/user/ams/hbase
and all the rest according to - https://cwiki.apache.org/confluence/display/AMBARI/AMS+-+distributed+mode
so ambari configured to save the data on hdfs instead the local disk
after metrics collector service restart we saw the following from log
2020-02-20 20:09:15,770 INFO [timeline] timeline.HadoopTimelineMetricsSink: No live collector to send metrics to. Metrics to be sent will be discarded. This message will be skipped for the next 20 times. 2020-02-20 20:11:35,769 INFO [timeline] timeline.HadoopTimelineMetricsSink: No live collector to send metrics to. Metrics to be sent will be discarded. This message will be skipped for the next 20 times.
and
2020-02-20 20:30:10,306 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: phoenix metrics system started 2020-02-20 20:30:10,450 WARN org.apache.hadoop.hbase.io.util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit is deprecated by hbase.regionserver.global.memstore.size 2020-02-20 20:30:15,327 WARN org.apache.hadoop.hbase.io.util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit is deprecated by hbase.regionserver.global.memstore.size
and without metrics as
and after some time the metrics collector service failed
why this happens ?
NOTE - I must to say that - when data of metrics saved on local disk then everything was ok
is that because reaching data on HDFS is more slow?
... View more
Labels:
02-12-2020
04:41 AM
hi all
we have HDP version - 2.6.4
with active and standby namenode
is it possible to capture the active namenode IP by REST API?
we try by the following API , but API not return any output
curl -sH "X-Requested-By: ambari" -u admin:admin -H "X-Requested-By: ambari" -X GET http://master:8080/api/v1/clusters//services/HDFS
... View more
Labels:
02-09-2020
06:29 PM
do you know , if some where we can found calculator for this parameter ? I mean lets say we have diff clusters and diff application running , and we want to tune this value according to our needs , then maybe some script that allow to set this value to the right tune?
... View more
02-09-2020
10:59 AM
we are running spark streaming app that consumes data from kafka topics
we want to know how to tune the parameter - `yarn.scheduler.capacity.maximum-am-resource-percent`
according to documentation:
yarn.scheduler.capacity.maximum-am-resource-percent: Maximum percent of resources in the cluster which can be used to run application masters i.e. controls number of concurrent running,
on some document we even see that recomneded to utilise it to `90 percent` for best results,
but the default is `10%`
so my question is
do we need to tune this parameter according to cluster size?
or what is the best practice in order to get good results?
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
02-03-2020
01:28 PM
1 Kudo
Dear Jay what to say excellent answer , you are really one of the best here
... View more
02-03-2020
11:32 AM
any way just for your information we get that curl -u admin:admin -H "X-Requested-By:ambari" -X GET "http://master_node:8080/api/v1/clusters/HDP/host_components?HostRoles/stale_configs=false&fields=HostRoles/service_name=HDFS" { "status" : 400, "message" : "Unable to compile query predicate: Invalid Query Token: token='=', previous token type=VALUE_OPERAND"
... View more
02-03-2020
11:23 AM
Dear Shelton , enjoy your vacation , and be careful from the corona -:)
... View more
02-03-2020
09:48 AM
I get A long list of all services , , any way not see there the HDFS but what we want is to verify only specific one service
... View more
02-03-2020
07:20 AM
but I am asking diff , we want API that ask about specific service as HDFS or YARN , etc this post that you mentioned ( https://community.cloudera.com/t5/Support-Questions/how-to-know-if-any-service-in-ambari-cluster-nee... ) is about all services that required restart note - what we want is API that ask the specific service if required restart or not
... View more
02-03-2020
06:52 AM
hi all
how to verify if specific service required restart?
for example - HDFS service
in case HDFS service required restart then ambari will show this as
is it possible by rest api to understand if service required restart or not?
for example for - YARN / HIVE , etc
... View more
Labels: