Member since
08-08-2017
1652
Posts
30
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1948 | 06-15-2020 05:23 AM | |
| 15858 | 01-30-2020 08:04 PM | |
| 2093 | 07-07-2019 09:06 PM | |
| 8186 | 01-27-2018 10:17 PM | |
| 4645 | 12-31-2017 10:12 PM |
06-08-2018
10:02 AM
how to validate that ? , I mean after I set in ambari "export JAVA_OPTS="-Djute.maxbuffer=11000000" and restart the zoo service then how to verify this seting is affected ? any way from some CLI to check that Djute.maxbuffer=11000000 ? we try: ps -ef | grep maxbuffer but no output, any suggestion how to validate the settings?
... View more
06-07-2018
09:17 PM
Geoffrey as I understand if I set the variable - export JAVA_OPTS="-Djute.maxbuffer=11000000" In ambari GUI ( as --> zookeper --> CONFIG --> zookeeper-env template ) then click on save and restart the service then it will be affected all zookeeper servers - am I right ? so no need to set it separate on each server,,
... View more
06-07-2018
07:51 PM
thank you Geoffrey , as I mentioned we want to increase only 0.1M to this value , as you said this isn't much , and we hope this will cause any issues second can you approve my syntax in zookeeper-env template in ambari? or maybe you have some remarks ? export JAVA_OPTS="-Djute.maxbuffer=11000000"
... View more
06-07-2018
04:52 PM
hi all when we upload conf file to zookeper we get the error - the .....xml weights 1095kb , which is more than zookeeper can have by default
(1024kb) so we decided to increase the ZNode size ( the default is limited to 1MB ) according to the article - https://community.hortonworks.com/content/supportkb/150660/error-javaioioexception-len-error-in-zookeeper-cau.html we set the following ( in order to support the 1095Kb size ) export JAVA_OPTS="-Djute.maxbuffer=11000000" in zookeeper-env template , ( ambari zookeeper CONFIG ) is this correct actions ?
... View more
Labels:
06-05-2018
03:59 PM
regarding the last option - Just block the public-repo-1.hortonworks.com public IP what you mean just "lock the public-repo-1.hortonworks.com public IP" HOW? remark we use only scripts to install ambari & HDP , so if we need to change someting in ambari we need to use API in order to perfrom some changing
... View more
06-05-2018
03:58 PM
regarding the /etc/hosts , do you mean to put as: 1.1.1.1 public-repo-1.hortonworks.com while no access to 1.1.1.1
... View more
06-05-2018
03:26 PM
how to disable ambari access to public-repo-1.hortonworks.com , or to disable network to block geting update from public-repo-1.hortonworks.com we want this in order to not get update from public-repo-1.hortonworks.com repo ping public-repo-1.hortonworks.com
64 bytes from server-52-85-183-236.fra50.r.cloudfront.net (52.85.183.236): icmp_seq=1 ttl=242 time=57.4 ms
64 bytes from server-52-85-183-236.fra50.r.cloudfront.net (52.85.183.236): icmp_seq=2 ttl=242 time=57.8 ms
64 bytes from server-52-85-183-236.fra50.r.cloudfront.net (52.85.183.236): icmp_seq=3 ttl=242 time=58.2 ms
64 bytes from server-52-85-183-236.fra50.r.cloudfront.net (52.85.183.236): icmp_seq=4 ttl=242 time=58.6 ms
64 bytes from server-52-85-183-236.fra50.r.cloudfront.net (52.85.183.236): icmp_seq=5 ttl=242 time=57.8 ms
... View more
Labels:
06-05-2018
10:04 AM
we have hadoop cluster version 2.6.4 while metrics collector installed on one of our Linux servers
root@master02 ambari-metrics-collector]# rpm -qa | grep metrics-collector
ambari-metrics-collector-2.6.1.0-143.x86_64
we have file of gc.log from Jul last year , and we want to automatic deletion of rotated gc.log files under /var/log/ambari-metrics-collector [root@master02 ambari-metrics-collector]# ls -ltr | grep gc.log | grep -v collector| more
-rw-r--r--. 1 ams hadoop 1430 Jul 13 2017 gc.log-201707131628
-rw-r--r-- 1 ams hadoop 178518767 Aug 28 2017 gc.log-201707160637
-rw-r--r-- 1 ams hadoop 128748829 Sep 17 2017 gc.log-201708280731
-rw-r--r-- 1 ams hadoop 60849 Sep 18 2017 gc.log-201709181421
-rw-r--r-- 1 ams hadoop 82562 Sep 18 2017 gc.log-201709181424
-rw-r--r-- 1 ams hadoop 164381669 Oct 19 2017 gc.log-201709181508
-rw-r--r-- 1 ams hadoop 46149 Oct 19 2017 gc.log-201710191738
-rw-r--r-- 1 ams hadoop 13763796 Oct 23 2017 gc.log-201710200633
-rw-r--r-- 1 ams hadoop 1048289 Oct 23 2017 gc.log-201710231021
-rw-r--r-- 1 ams hadoop 3473874 Oct 24 2017 gc.log-201710231649
-rw-r--r-- 1 ams hadoop 137289030 Nov 15 2017 gc.log-201710241352
-rw-r--r-- 1 ams hadoop 8037548 Nov 16 2017 gc.log-201711151355
-rw-r--r-- 1 ams hadoop 23919164 Nov 19 2017 gc.log-201711161246
-rw-r--r-- 1 ams hadoop 139759594 Dec 6 16:35 gc.log-201711191241
-rw-r--r-- 1 ams hadoop 184118442 Dec 25 16:13 gc.log-201712061635
-rw-r--r-- 1 ams hadoop 1440 Jan 1 04:38 gc.log-201712251613
-rw-r--r-- 1 ams hadoop 1437 Jan 10 00:03 gc.log-201801021532
-rw-r--r-- 1 ams hadoop 1439 Jan 10 01:04 gc.log-201801100004
-rw-r--r-- 1 ams hadoop 3637 Jan 10 01:11 gc.log-201801100105
-rw-r--r-- 1 ams hadoop 1437 Jan 10 01:16 gc.log-201801100113
-rw-r--r-- 1 ams hadoop 3636 Jan 10 01:24 gc.log-201801100118
-rw-r--r-- 1 ams hadoop 1438 Jan 10 01:35 gc.log-201801100126
-rw-r--r-- 1 ams hadoop 1438 Jan 10 01:42 gc.log-201801100137
-rw-r--r-- 1 ams hadoop 1436 Jan 10 02:14 gc.log-201801100144
-rw-r--r-- 1 ams hadoop 8353 Jan 10 02:19 gc.log-201801100216
-rw-r--r-- 1 ams hadoop 1437 Jan 11 09:08 gc.log-201801110730
-rw-r--r-- 1 ams hadoop 1436 Jan 11 09:19 gc.log-201801110909
-rw-r--r-- 1 ams hadoop 766 Jan 11 09:20 gc.log-201801110920
-rw-r--r-- 1 ams hadoop 765 Jan 11 09:26 gc.log-201801110926 from Advanced ams-hbase-env in ambari gui we see the following : {% if java_version < 8 %}
export HBASE_MASTER_OPTS=" -XX:PermSize=64m -XX:MaxPermSize={{hbase_master_maxperm_size}} -Xms{{hbase_heapsize}} -Xmx{{hbase_heapsize}} -Xmn{{hbase_master_xmn_size}} -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly"
export HBASE_REGIONSERVER_OPTS="-XX:MaxPermSize=128m -Xmn{{regionserver_xmn_size}} -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Xms{{regionserver_heapsize}} -Xmx{{regionserver_heapsize}}"
{% else %}
export HBASE_MASTER_OPTS=" -Xms{{hbase_heapsize}} -Xmx{{hbase_heapsize}} -Xmn{{hbase_master_xmn_size}} -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly"
export HBASE_REGIONSERVER_OPTS=" -Xmn{{regionserver_xmn_size}} -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Xms{{regionserver_heapsize}} -Xmx{{regionserver_heapsize}}"
{% endif %} and we try to set in HBASE_MASTER_OPTS or HBASE_REGIONSERVER_OPTS
the **-XX:NumberOfGClogFiles=10** in order to get only 10 gc.log files , but metrics collector failed to start , seems because incorrect syntax
export HBASE_MASTER_OPTS=" -XX:PermSize=64m -XX:MaxPermSize={{hbase_master_maxperm_size}} -Xms{{hbase_heapsize}} -Xmx{{hbase_heapsize}} -Xmn{{hbase_master_xmn_size}} -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:NumberOfGClogFiles=1" please advice how to perform automatic deletion of rotated gc.log files
... View more
Labels:
06-04-2018
08:13 PM
how to limit the logs rotation under /var/log/ambari-metrics-collector so logs will be only on the last month under /var/log/ambari-metrics-collector we see logs from Jul , but we want only logs until month ( logs rotation no longer then month ) otal 7277584
-rw-r--r--. 1 ams hadoop 13794 Jul 13 2017 collector-gc.log-201707131628
-rw-r--r--. 1 ams hadoop 1430 Jul 13 2017 gc.log-201707131628
-rw-r--r-- 1 ams hadoop 126748287 Aug 14 2017 collector-gc.log-201707160637
-rw-r--r-- 1 ams hadoop 178518767 Aug 28 2017 gc.log-201707160637
-rw-r--r--. 1 ams hadoop 83886155 Sep 14 2017 ambari-metrics-collector.log.30
-rw-r--r-- 1 ams hadoop 94522724 Sep 17 2017 collector-gc.log-201708280731
-rw-r--r-- 1 ams hadoop 128748829 Sep 17 2017 gc.log-201708280731
-rw-r--r-- 1 ams hadoop 2850 Sep 18 2017 collector-gc.log-201709181421
-rw-r--r-- 1 ams hadoop 60849 Sep 18 2017 gc.log-201709181421
-rw-r--r-- 1 ams hadoop 3517 Sep 18 2017 collector-gc.log-201709181424
-rw-r--r-- 1 ams hadoop 82562 Sep 18 2017 gc.log-201709181424
-rw-r--r-- 1 ams hadoop 117495363 Oct 19 2017 collector-gc.log-201709181508
-rw-r--r-- 1 ams hadoop 164381669 Oct 19 2017 gc.log-201709181508
-rw-r--r-- 1 ams hadoop 16634 Oct 19 2017 collector-gc.log-201710191738
-rw-r--r-- 1 ams hadoop 46149 Oct 19 2017 gc.log-201710191738
-rw-r--r-- 1 ams hadoop 7052909 Oct 23 2017 collector-gc.log-201710200633
-rw-r--r-- 1 ams hadoop 13763796 Oct 23 2017 gc.log-201710200633
-rw-r--r-- 1 ams hadoop 632288 Oct 23 2017 collector-gc.log-201710231021
-rw-r--r-- 1 ams hadoop 1048289 Oct 23 2017 gc.log-201710231021
-rw-r--r-- 1 ams hadoop 1927171 Oct 24 2017 collector-gc.log-201710231649
-rw-r--r-- 1 ams hadoop 3473874 Oct 24 2017 gc.log-201710231649
-rw-r--r--. 1 ams hadoop 268435531 Oct 28 2017 hbase-ams-master-master02.sys654.com.log.5
-rw-r--r-- 1 ams hadoop 83886115 Nov 1 2017 ambari-metrics-collector.log.29
-rw-r--r-- 1 ams hadoop 79689213 Nov 15 2017 collector-gc.log-201710241352
-rw-r--r-- 1 ams hadoop 137289030 Nov 15 2017 gc.log-201710241352
-rw-r--r-- 1 ams hadoop 4736994 Nov 16 2017 collector-gc.log-201711151355
-rw-r--r-- 1 ams hadoop 8037548 Nov 16 2017 gc.log-201711151355
-rw-r--r-- 1 ams hadoop 15168598 Nov 19 2017 collector-gc.log-201711161246
-rw-r--r-- 1 ams hadoop 23919164 Nov 19 2017 gc.log-201711161246
-rw-r--r-- 1 ams hadoop 83720006 Dec 6 16:32 collector-gc.log-201711191241
-rw-r--r-- 1 ams hadoop 139759594 Dec 6 16:35 gc.log-201711191241
-rw-r--r-- 1 ams hadoop 83886082 Dec 17 10:03 ambari-metrics-collector.log.28
-rw-r--r-- 1 ams hadoop 110573406 Dec 25 16:10 collector-gc.log-201712061635
-rw-r--r-- 1 ams hadoop 184118442 Dec 25 16:13 gc.log-201712061635
-rw-r--r-- 1 ams hadoop 1440 Jan 1 04:38 gc.log-201712251613
-rw-r--r-- 1 ams hadoop 39360581 Jan 1 04:40 collector-gc.log-201712251613
-rw-r--r-- 1 ams hadoop 1437 Jan 10 00:03 gc.log-201801021532
-rw-r--r-- 1 ams hadoop 42005865 Jan 10 00:04 collector-gc.log-201801021532
-rw-r--r-- 1 ams hadoop 1439 Jan 10 01:04 gc.log-201801100004
-rw-r--r-- 1 ams hadoop 286342 Jan 10 01:05 collector-gc.log-201801100004
-rw-r--r-- 1 ams hadoop 3637 Jan 10 01:11 gc.log-201801100105
-rw-r--r-- 1 ams hadoop 3085 Jan 10 01:13 collector-gc.log-201801100105
-rw-r--r-- 1 ams hadoop 1437 Jan 10 01:16 gc.log-201801100113
-rw-r--r-- 1 ams hadoop 7230 Jan 10 01:18 collector-gc.log-201801100113
-rw-r--r-- 1 ams hadoop 3636 Jan 10 01:24 gc.log-201801100118
-rw-r--r-- 1 ams hadoop 3085 Jan 10 01:25 collector-gc.log-201801100118
-rw-r--r-- 1 ams hadoop 1438 Jan 10 01:35 gc.log-201801100126
-rw-r--r-- 1 ams hadoop 17357 Jan 10 01:36 collector-gc.log-201801100126
... View more
Labels:
06-04-2018
12:04 PM
who is the "system administrator." , in that case ?
... View more