Member since
03-21-2018
20
Posts
0
Kudos Received
0
Solutions
05-02-2018
07:05 AM
@Jay Kumar SenSharma Thanks for reply !! Do we have any recommended documentation about what should be the idle backup log file, retention days ...etc.
... View more
05-01-2018
12:08 PM
I am looking for ambari metrics logging policies.I found some of the links in hortonworks community https://community.hortonworks.com/questions/85120/ambari-metrics-collector-logging.html https://community.hortonworks.com/articles/50058/using-log4j-extras-how-to-rotate-as-well-as-zip-th.html https://logging.apache.org/log4j/extras/apidocs/org/apache/log4j/rolling/ I am looking for other logging properties mentioned in ambari metrics which is not mentioned in above links. for e.g, log4j.appender.DRFA, log4j.appender.RFA. What does all log properties means under ambari metrics -> config --> Advanced log4j. What are the recommended properties for ambari metrics. Retention policy of logging. Do any one have any link from horton works community which can explain. Appreciate your support. Thanks!
... View more
Labels:
- Labels:
-
Apache Ambari
04-25-2018
01:33 PM
What all logging goes in /var/log/messages. Where we can apply adjustment to stop unwanted logging in it. How we can apply retention to it.
... View more
- Tags:
- Hadoop Core
- logs
04-19-2018
12:16 PM
@Jay Kumar SenSharma For example , I have two hosts in a cluster hostA, HostB HostA is on rack-1 and HostB is on rack-2 Services components running in hostA are HDFS_CLINET, ZOOKEEPER,NAMENODE..etc Services components running in hostB are HBASE, AMABRI_METRICS,DATANODE...etc I want get result from api like Host: HostA, Service_component : NAMENODE, rack_info: rack-1 HostA, Service_component : ZOOKEEPER, rack_info: rack-1 HostB, Service_component : DATANODE, rack_info: rack-2 --------------------------------------
... View more
04-19-2018
12:03 PM
@Jay Kumar SenSharma
Thanks Jay!! My requirement to check each service get host name and rack info.Can we achieve this in one api. Appreciate your help.
... View more
04-19-2018
11:23 AM
Do we have any api to get rack info of each /service in an ambari cluster
... View more
Labels:
- Labels:
-
Apache Ambari
04-13-2018
02:08 PM
@Jay Kumar SenSharma Thanks for the reply!! I am setting database in other than ambari-server (other host). How to do that setup. How ambari will know which database needs to be connected. Please help Regards.
... View more
04-13-2018
12:42 PM
I want to use external ( location other than ambari-server host) database (MariaDB or Oracle 11g) for Ambari. Can any one help me to know the steps to be followed.
... View more
Labels:
- Labels:
-
Apache Ambari
04-12-2018
09:14 AM
@Jay Kumar SenSharma Please find the requested details 1. ambari server and ams binary versions $ rpm -qa | grep ambari ambari-metrics-monitor-2.5.0.3-7.x86_64 ambari-metrics-grafana-2.5.0.3-7.x86_64 ambari-metrics-hadoop-sink-2.5.0.3-7.x86_64 ambari-agent-2.5.0.3-7.x86_64 ambari-metrics-collector-2.5.0.3-7.x86_64 2. ambari-metrics monitors are pointing to correct servers metrics_servers = ics024045196.ics-eu-1.asml.com,ics024045188.ics-eu-1.asml.com 3. $ nc -v ics024045188.ics-eu-1.asml.com 6188 Ncat: Version 6.40 ( http://nmap.org/ncat ) Ncat: Connected to 172.24.45.188:6188. $ telnet ics024045188.ics-eu-1.asml.com 6188 Trying 172.24.45.188... Connected to ics024045188.ics-eu-1.asml.com. 4. No, I didnt upgarded amabri metrics 5. ambari-monitor logs 2017-11-22 11:24:04,908 [INFO] host_info.py:352 - Cached hostname: ics024045188.ics-eu-1.asml.com 2017-11-22
11:24:04,909 [INFO] controller.py:101 - Adding event to cache, all :
{u'metrics': [{u'value_threshold': u'128', u'name': u'bytes_out'}],
u'collect_every': u'10'} 2017-11-22 11:24:04,909 [INFO] main.py:65 -
Starting Server RPC Thread:
/usr/lib/python2.6/site-packages/resource_monitoring/main.py start 2017-11-22 11:24:04,909 [INFO] controller.py:55 - Running Controller thread: Thread-1 2017-11-22 11:24:04,909 [INFO] emitter.py:60 - Running Emitter thread: Thread-2 2017-11-22 11:24:04,910 [INFO] emitter.py:79 - Nothing to emit, resume waiting. 2017-11-22 11:25:04,912 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com 2017-11-22 11:26:04,919 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com 2017-11-22 11:27:04,930 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com 2017-11-22 11:28:04,939 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com 2017-11-22 11:29:04,953 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com 2017-11-22 11:30:04,968 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com 2017-11-22 11:31:04,994 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com 2017-11-22 11:32:05,039 [INFO] emitter.py:152 - Calculated collector shard based on hostname : ics024045196.ics-eu-1.asml.com Server out at: /var/log/ambari-metrics-host-monitoring/ambari-metrics-host-monitoring.out Server log at: /var/log/ambari-metrics-host-monitoring/ambari-metrics-host-monitoring.log
... View more
04-12-2018
07:10 AM
batch4-ambarimetrics.jpgbatch4-hosts-metrics.jpg I have migrated metrics collector data and changed metrics collector embedded mode to distributed for that I referred https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-operations/content/ams_metrics_collector_modes.html link . There are no errors in logs. Everything seems to be fine except I cannot see Ambari Metrics and Hosts Metrics. Attached the screenshot. Thanks!!
... View more
Labels:
- Labels:
-
Apache Ambari
04-06-2018
02:39 PM
I posted the question very early. Steps to be followed sudo su hdfs hdfs dfs -ls /var/lib/ambari-metrics-collector/hbase/data/default
... View more
04-06-2018
02:07 PM
I have install two ambari-metrics-collector in Ambari cluster using distributed mode. I have set below properties in blueprint "ams-site" : { "properties_attributes" : { }, "properties" : { "timeline.metrics.service.operation.mode" : "distributed" }}}, { "ams-hbase-site" : { "properties_attributes" : { }, "properties" : { "hbase.rootdir" : "hdfs://<<NameserviceId>>/var/lib/ambari-metrics-collector/hbase", "hbase.cluster.distributed" : "true", "dfs.client.read.shortcircuit" : "false" }}} I want to know steps . How to check data of ambari-metrics-collector in hdfs. Appreciate any help!!
... View more
Labels:
- Labels:
-
Apache Ambari
04-04-2018
12:19 PM
I wan to install two metrics grafana in two different nodes for high availibility. I haven't tried it but looking some references and experts advise. Thanks!!
... View more
Labels:
- Labels:
-
Apache Ambari
03-26-2018
03:12 PM
I set below properties in blueprint still its creating some groups and users { "kafka-env" : { "properties_attributes" : { }, "properties" : { "kafka_user" : "{{ ambari_user_prefix }}-kafka" }}}, { "ams-env" : { "properties_attributes" : { }, "properties" : { "ambari_metrics_user" : "{{ ambari_user_prefix }}-ams" }}}, { "zookeeper-env" : { "properties_attributes" : { }, "properties" : { "zk_user" : "{{ ambari_user_prefix }}-zookeeper" }}}, { "cluster-env" : { "properties_attributes" : { }, "properties" : { "user_group" : "{{ ambari_user_prefix }}-hadoop", "smokeuser" : "{{ ambari_user_prefix }}-ambari-qa" }},{ "hbase-env" : { "properties_attributes" : { }, "properties" : { "hbase_user" : "{{ ambari_user_prefix }}-hbase", }}}, { "hadoop-env" : { "properties_attributes" : { }, "properties" : { "hdfs_user" : "{{ ambari_user_prefix }}-hdfs", }}}, { "hdfs-site" : { "properties" : { "dfs.permissions.superusergroup" : "{{ ambari_user_prefix }}-hdfs", "dfs.cluster.administrators" : " {{ ambari_user_prefix }}-hdfs", "dfs.block.local-path-access.user" : "{{ ambari_user_prefix }}-hbase" }} { "kafka-broker" : { "properties" : { "kafka.ganglia.metrics.group" : "{{ ambari_user_prefix }}-kafka" } { "core-site" : { "properties" : { "hadoop.proxyuser.{{ ambari_user_prefix }}-hbase.hosts" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-hbase.groups" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-hdfs.groups" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-hdfs.hosts" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-kafka.groups" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-kafka.hosts" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-zookeeper.groups" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-zookeeper.hosts" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-mapred.groups" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-mapred.hosts" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-yarn.groups" : "*", "hadoop.proxyuser.{{ ambari_user_prefix }}-yarn.hosts" : "*", "fs.defaultFS" : "hdfs://{{ ambari_cluster_name }}", "ha.zookeeper.quorum" : "%HOSTGROUP::master_0%:12181,%HOSTGROUP::master_1%:12181,%HOSTGROUP::master_2%:12181" } Groups created vcp-9999-hdfs:x:16031:vcp-9999-hdfs vcp-9999-hadoop:x:16032:vcp-9999-zookeeper,vcp-9999-hbase,vcp-9999-kafka,vcp-9999-ams,vcp-9999-hdfs zookeeper:x:498: hadoop:x:497: yarn:x:496: mapred:x:495: hdfs:x:494: Users created vcp-9999-zookeeper:x:16028:16032::/home/vcp-9999-zookeeper:/bin/bash vcp-9999-hbase:x:16031:16032::/home/vcp-9999-hbase:/bin/bash vcp-9999-ambari-qa:x:16034:16032::/home/vcp-9999-ambari-qa:/bin/bash vcp-9999-kafka:x:16035:16032::/home/vcp-9999-kafka:/bin/bash vcp-9999-ams:x:16038:16032::/home/vcp-9999-ams:/bin/bash vcp-9999-hdfs:x:16039:16032::/home/vcp-9999-hdfs:/bin/bash kafka:x:995:492:KAFKA:/home/kafka:/bin/bash Can any one help to explain that why groups and users created ( in bold text) even though I have added properties. How can we remove these users at the time of creation. Thanks!
... View more
03-22-2018
10:43 AM
Thanks All !! Let me try...Appreciate your help.
... View more