Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

ambari metrics collector not starting on a node

Labels:
avatar
author-rank SSandhu
New Contributor

Created ‎04-02-2023 12:05 PM

Tried to add ambari metrics to a cluster and the metrics collector keeps failing on a node due to some issue


here are the logs:

stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/AMBARI_METRICS/package/scripts/metrics_collector.py", line 90, in <module>
AmsCollector().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/AMBARI_METRICS/package/scripts/metrics_collector.py", line 51, in start
ams_service('collector', action = 'start')
File "/usr/lib/ambari-agent/lib/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/AMBARI_METRICS/package/scripts/ams_service.py", line 84, in ams_service
user=params.ams_user
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf start' returned 255. Sun Apr 2 23:39:41 IST 2023 Starting HBase.
running master, logging to /var/log/ambari-metrics-collector/hbase-ams-master-hdp4000.sre.iti.acceldata.dev.out
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/phoenix-5.0.0.3.1.4.0-313-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Verifying ambari-metrics-collector process status...
ERROR: ambari-metrics-collector start failed. For more details, see /var/log/ambari-metrics-collector/ambari-metrics-collector.out:
====================
Error: Could not find or load main class org.apache.ambari.metrics.AMSApplicationServer
====================
Collector out at: /var/log/ambari-metrics-collector/ambari-metrics-collector.out
stdout:
2023-04-02 23:39:32,894 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.4.0-315 -> 3.1.4.0-315
2023-04-02 23:39:32,929 - Using hadoop conf dir: /usr/hdp/3.1.4.0-315/hadoop/conf
2023-04-02 23:39:33,345 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.4.0-315 -> 3.1.4.0-315
2023-04-02 23:39:33,355 - Using hadoop conf dir: /usr/hdp/3.1.4.0-315/hadoop/conf
2023-04-02 23:39:33,357 - Group['livy'] {}
2023-04-02 23:39:33,359 - Group['spark'] {}
2023-04-02 23:39:33,359 - Group['ranger'] {}
2023-04-02 23:39:33,360 - Group['hdfs'] {}
2023-04-02 23:39:33,360 - Group['zeppelin'] {}
2023-04-02 23:39:33,360 - Group['hadoop'] {}
2023-04-02 23:39:33,361 - Group['users'] {}
2023-04-02 23:39:33,362 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2023-04-02 23:39:33,363 - User['yarn-ats'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2023-04-02 23:39:33,365 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2023-04-02 23:39:33,366 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2023-04-02 23:39:33,367 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2023-04-02 23:39:33,369 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['ranger', 'hadoop'], 'uid': None}
2023-04-02 23:39:33,370 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2023-04-02 23:39:33,371 - User['zeppelin'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['zeppelin', 'hadoop'], 'uid': None}
2023-04-02 23:39:33,372 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['livy', 'hadoop'], 'uid': None}
2023-04-02 23:39:33,373 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['spark', 'hadoop'], 'uid': None}
2023-04-02 23:39:33,375 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2023-04-02 23:39:33,376 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop'], 'uid': None}
2023-04-02 23:39:33,377 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2023-04-02 23:39:33,379 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2023-04-02 23:39:33,380 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2023-04-02 23:39:33,382 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2023-04-02 23:39:33,395 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2023-04-02 23:39:33,396 - Group['hdfs'] {}
2023-04-02 23:39:33,397 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop', u'hdfs']}
2023-04-02 23:39:33,398 - FS Type: HDFS
2023-04-02 23:39:33,398 - Directory['/etc/hadoop'] {'mode': 0755}
2023-04-02 23:39:33,425 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2023-04-02 23:39:33,427 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2023-04-02 23:39:33,471 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2023-04-02 23:39:33,499 - Skipping Execute[('setenforce', '0')] due to only_if
2023-04-02 23:39:33,500 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'}
2023-04-02 23:39:33,504 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'}
2023-04-02 23:39:33,505 - Directory['/var/run/hadoop/hdfs'] {'owner': 'hdfs', 'cd_access': 'a'}
2023-04-02 23:39:33,507 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'}
2023-04-02 23:39:33,513 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2023-04-02 23:39:33,516 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2023-04-02 23:39:33,525 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2023-04-02 23:39:33,543 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2023-04-02 23:39:33,545 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2023-04-02 23:39:33,546 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2023-04-02 23:39:33,553 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop', 'mode': 0644}
2023-04-02 23:39:33,564 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2023-04-02 23:39:33,574 - Skipping unlimited key JCE policy check and setup since it is not required
2023-04-02 23:39:33,586 - Skipping stack-select on AMBARI_METRICS because it does not exist in the stack-select package structure.
2023-04-02 23:39:34,163 - Using hadoop conf dir: /usr/hdp/3.1.4.0-315/hadoop/conf
2023-04-02 23:39:34,166 - checked_call['hostid'] {}
2023-04-02 23:39:34,178 - checked_call returned (0, '5a0a4706')
2023-04-02 23:39:34,185 - Directory['/etc/ams-hbase/conf'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True}
2023-04-02 23:39:34,190 - Directory['/var/lib/ambari-metrics-collector/hbase-tmp'] {'owner': 'ams', 'create_parents': True, 'recursive_ownership': True, 'cd_access': 'a'}
2023-04-02 23:39:34,194 - Directory['/var/lib/ambari-metrics-collector/hbase-tmp/local/jars'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}
2023-04-02 23:39:34,196 - File['/etc/ams-hbase/conf/core-site.xml'] {'owner': 'ams', 'action': ['delete']}
2023-04-02 23:39:34,197 - File['/etc/ams-hbase/conf/hdfs-site.xml'] {'owner': 'ams', 'action': ['delete']}
2023-04-02 23:39:34,197 - XmlConfig['hbase-site.xml'] {'owner': 'ams', 'group': 'hadoop', 'conf_dir': '/etc/ams-hbase/conf', 'configuration_attributes': {u'final': {u'hbase.zookeeper.quorum': u'true'}}, 'configurations': ...}
2023-04-02 23:39:34,227 - Generating config: /etc/ams-hbase/conf/hbase-site.xml
2023-04-02 23:39:34,227 - File['/etc/ams-hbase/conf/hbase-site.xml'] {'owner': 'ams', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2023-04-02 23:39:34,316 - XmlConfig['hbase-policy.xml'] {'owner': 'ams', 'group': 'hadoop', 'conf_dir': '/etc/ams-hbase/conf', 'configuration_attributes': {}, 'configurations': {u'security.admin.protocol.acl': u'*', u'security.masterregion.protocol.acl': u'*', u'security.client.protocol.acl': u'*'}}
2023-04-02 23:39:34,339 - Generating config: /etc/ams-hbase/conf/hbase-policy.xml
2023-04-02 23:39:34,340 - File['/etc/ams-hbase/conf/hbase-policy.xml'] {'owner': 'ams', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2023-04-02 23:39:34,367 - File['/etc/ams-hbase/conf/hbase-env.sh'] {'content': InlineTemplate(...), 'owner': 'ams'}
2023-04-02 23:39:34,385 - File['/etc/ams-hbase/conf/hadoop-metrics2-hbase.properties'] {'content': Template('hadoop-metrics2-hbase.properties.j2'), 'owner': 'ams', 'group': 'hadoop'}
2023-04-02 23:39:34,386 - TemplateConfig['/etc/ams-hbase/conf/regionservers'] {'owner': 'ams', 'template_tag': None}
2023-04-02 23:39:34,391 - File['/etc/ams-hbase/conf/regionservers'] {'content': Template('regionservers.j2'), 'owner': 'ams', 'group': None, 'mode': None}
2023-04-02 23:39:34,393 - Directory['/var/run/ambari-metrics-collector/'] {'owner': 'ams', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,394 - Directory['/var/log/ambari-metrics-collector'] {'owner': 'ams', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,395 - Directory['/var/lib/ambari-metrics-collector/hbase'] {'owner': 'ams', 'create_parents': True, 'recursive_ownership': True, 'cd_access': 'a'}
2023-04-02 23:39:34,404 - File['/var/run/ambari-metrics-collector//distributed_mode'] {'owner': 'ams', 'action': ['delete']}
2023-04-02 23:39:34,412 - File['/etc/ams-hbase/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'ams', 'group': 'hadoop', 'mode': 0644}
2023-04-02 23:39:34,426 - Directory['/usr/lib/ambari-logsearch-logfeeder/conf'] {'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,427 - Generate Log Feeder config file: /usr/lib/ambari-logsearch-logfeeder/conf/input.config-ambari-metrics.json
2023-04-02 23:39:34,427 - File['/usr/lib/ambari-logsearch-logfeeder/conf/input.config-ambari-metrics.json'] {'content': Template('input.config-ambari-metrics.json.j2'), 'mode': 0644}
2023-04-02 23:39:34,430 - Directory['/etc/ams-hbase/conf'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True}
2023-04-02 23:39:34,431 - Directory['/var/lib/ambari-metrics-collector/hbase-tmp'] {'owner': 'ams', 'create_parents': True, 'recursive_ownership': True, 'cd_access': 'a'}
2023-04-02 23:39:34,434 - Directory['/var/lib/ambari-metrics-collector/hbase-tmp/local/jars'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}
2023-04-02 23:39:34,435 - File['/etc/ams-hbase/conf/core-site.xml'] {'owner': 'ams', 'action': ['delete']}
2023-04-02 23:39:34,436 - File['/etc/ams-hbase/conf/hdfs-site.xml'] {'owner': 'ams', 'action': ['delete']}
2023-04-02 23:39:34,437 - XmlConfig['hbase-site.xml'] {'owner': 'ams', 'group': 'hadoop', 'conf_dir': '/etc/ams-hbase/conf', 'configuration_attributes': {u'final': {u'hbase.zookeeper.quorum': u'true'}}, 'configurations': ...}
2023-04-02 23:39:34,458 - Generating config: /etc/ams-hbase/conf/hbase-site.xml
2023-04-02 23:39:34,459 - File['/etc/ams-hbase/conf/hbase-site.xml'] {'owner': 'ams', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2023-04-02 23:39:34,545 - XmlConfig['hbase-policy.xml'] {'owner': 'ams', 'group': 'hadoop', 'conf_dir': '/etc/ams-hbase/conf', 'configuration_attributes': {}, 'configurations': {u'security.admin.protocol.acl': u'*', u'security.masterregion.protocol.acl': u'*', u'security.client.protocol.acl': u'*'}}
2023-04-02 23:39:34,567 - Generating config: /etc/ams-hbase/conf/hbase-policy.xml
2023-04-02 23:39:34,568 - File['/etc/ams-hbase/conf/hbase-policy.xml'] {'owner': 'ams', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2023-04-02 23:39:34,585 - File['/etc/ams-hbase/conf/hbase-env.sh'] {'content': InlineTemplate(...), 'owner': 'ams'}
2023-04-02 23:39:34,594 - File['/etc/ams-hbase/conf/hadoop-metrics2-hbase.properties'] {'content': Template('hadoop-metrics2-hbase.properties.j2'), 'owner': 'ams', 'group': 'hadoop'}
2023-04-02 23:39:34,596 - TemplateConfig['/etc/ams-hbase/conf/regionservers'] {'owner': 'ams', 'template_tag': None}
2023-04-02 23:39:34,598 - File['/etc/ams-hbase/conf/regionservers'] {'content': Template('regionservers.j2'), 'owner': 'ams', 'group': None, 'mode': None}
2023-04-02 23:39:34,600 - Directory['/var/run/ambari-metrics-collector/'] {'owner': 'ams', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,600 - Directory['/var/log/ambari-metrics-collector'] {'owner': 'ams', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,605 - File['/etc/ams-hbase/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'ams', 'group': 'hadoop', 'mode': 0644}
2023-04-02 23:39:34,620 - Directory['/usr/lib/ambari-logsearch-logfeeder/conf'] {'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,621 - Generate Log Feeder config file: /usr/lib/ambari-logsearch-logfeeder/conf/input.config-ambari-metrics.json
2023-04-02 23:39:34,621 - File['/usr/lib/ambari-logsearch-logfeeder/conf/input.config-ambari-metrics.json'] {'content': Template('input.config-ambari-metrics.json.j2'), 'mode': 0644}
2023-04-02 23:39:34,623 - Directory['/etc/ambari-metrics-collector/conf'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True}
2023-04-02 23:39:34,624 - Directory['/var/lib/ambari-metrics-collector/checkpoint'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'recursive_ownership': True, 'cd_access': 'a'}
2023-04-02 23:39:34,626 - XmlConfig['ams-site.xml'] {'owner': 'ams', 'group': 'hadoop', 'conf_dir': '/etc/ambari-metrics-collector/conf', 'configuration_attributes': {}, 'configurations': ...}
2023-04-02 23:39:34,640 - Generating config: /etc/ambari-metrics-collector/conf/ams-site.xml
2023-04-02 23:39:34,640 - File['/etc/ambari-metrics-collector/conf/ams-site.xml'] {'owner': 'ams', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2023-04-02 23:39:34,737 - XmlConfig['ssl-server.xml'] {'owner': 'ams', 'group': 'hadoop', 'conf_dir': '/etc/ambari-metrics-collector/conf', 'configuration_attributes': {}, 'configurations': ...}
2023-04-02 23:39:34,756 - Generating config: /etc/ambari-metrics-collector/conf/ssl-server.xml
2023-04-02 23:39:34,757 - File['/etc/ambari-metrics-collector/conf/ssl-server.xml'] {'owner': 'ams', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2023-04-02 23:39:34,775 - XmlConfig['hbase-site.xml'] {'owner': 'ams', 'group': 'hadoop', 'conf_dir': '/etc/ambari-metrics-collector/conf', 'configuration_attributes': {u'final': {u'hbase.zookeeper.quorum': u'true'}}, 'configurations': ...}
2023-04-02 23:39:34,797 - Generating config: /etc/ambari-metrics-collector/conf/hbase-site.xml
2023-04-02 23:39:34,798 - File['/etc/ambari-metrics-collector/conf/hbase-site.xml'] {'owner': 'ams', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2023-04-02 23:39:34,883 - File['/etc/ambari-metrics-collector/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'ams', 'group': 'hadoop', 'mode': 0644}
2023-04-02 23:39:34,890 - File['/etc/ambari-metrics-collector/conf/ams-env.sh'] {'content': InlineTemplate(...), 'owner': 'ams'}
2023-04-02 23:39:34,891 - Directory['/var/log/ambari-metrics-collector'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,892 - Directory['/var/run/ambari-metrics-collector'] {'owner': 'ams', 'group': 'hadoop', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,892 - File['/usr/lib/ams-hbase/bin/hadoop'] {'owner': 'ams', 'mode': 0755}
2023-04-02 23:39:34,893 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2023-04-02 23:39:34,897 - File['/etc/security/limits.d/ams.conf'] {'content': Template('ams.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2023-04-02 23:39:34,909 - Directory['/usr/lib/ambari-logsearch-logfeeder/conf'] {'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2023-04-02 23:39:34,910 - Generate Log Feeder config file: /usr/lib/ambari-logsearch-logfeeder/conf/input.config-ambari-metrics.json
2023-04-02 23:39:34,910 - File['/usr/lib/ambari-logsearch-logfeeder/conf/input.config-ambari-metrics.json'] {'content': Template('input.config-ambari-metrics.json.j2'), 'mode': 0644}
2023-04-02 23:39:34,913 - Execute['/usr/lib/ams-hbase/bin/hbase-daemon.sh --config /etc/ams-hbase/conf stop regionserver'] {'on_timeout': 'ls /var/run/ambari-metrics-collector//hbase-ams-regionserver.pid >/dev/null 2>&1 && ps `cat /var/run/ambari-metrics-collector//hbase-ams-regionserver.pid` >/dev/null 2>&1 && ambari-sudo.sh -H -E kill -9 `ambari-sudo.sh cat /var/run/ambari-metrics-collector//hbase-ams-regionserver.pid`', 'timeout': 30, 'user': 'ams'}
2023-04-02 23:39:35,115 - File['/var/run/ambari-metrics-collector//hbase-ams-regionserver.pid'] {'action': ['delete']}
2023-04-02 23:39:35,116 - Execute['/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf stop'] {'user': 'ams'}
2023-04-02 23:39:41,395 - Execute['ambari-sudo.sh rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/*.tmp'] {}
2023-04-02 23:39:41,413 - File['/etc/ambari-metrics-collector/conf/core-site.xml'] {'owner': 'ams', 'action': ['delete']}
2023-04-02 23:39:41,414 - File['/etc/ambari-metrics-collector/conf/hdfs-site.xml'] {'owner': 'ams', 'action': ['delete']}
2023-04-02 23:39:41,414 - Execute['/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf start'] {'user': 'ams'}
2023-04-02 23:39:44,684 - Execute['find /var/log/ambari-metrics-collector -maxdepth 1 -type f -name '*' -exec echo '==> {} <==' \; -exec tail -n 40 {} \;'] {'logoutput': True, 'ignore_failures': True, 'user': 'ams'}
==> /var/log/ambari-metrics-collector/ambari-metrics-collector-startup.out <==
Stopping HBase master
Mon Jan 23 14:09:55 IST 2023 Starting HBase.
Verifying ambari-metrics-collector process status...
ERROR: ambari-metrics-collector start failed. For more details, see /var/log/ambari-metrics-collector/ambari-metrics-collector.out:
Collector out at: /var/log/ambari-metrics-collector/ambari-metrics-collector.out
Stopping HBase master
Sun Apr 2 23:35:38 IST 2023 Starting HBase.
Verifying ambari-metrics-collector process status...
ERROR: ambari-metrics-collector start failed. For more details, see /var/log/ambari-metrics-collector/ambari-metrics-collector.out:
Collector out at: /var/log/ambari-metrics-collector/ambari-metrics-collector.out
Stopping HBase master
master is running as process 90026. Continuing
Sun Apr 2 23:35:51 IST 2023 Starting HBase.
Verifying ambari-metrics-collector process status...
ERROR: ambari-metrics-collector start failed. For more details, see /var/log/ambari-metrics-collector/ambari-metrics-collector.out:
Collector out at: /var/log/ambari-metrics-collector/ambari-metrics-collector.out
Stopping HBase master
master is running as process 90463. Continuing
Sun Apr 2 23:39:41 IST 2023 Starting HBase.
Verifying ambari-metrics-collector process status...
ERROR: ambari-metrics-collector start failed. For more details, see /var/log/ambari-metrics-collector/ambari-metrics-collector.out:
Collector out at: /var/log/ambari-metrics-collector/ambari-metrics-collector.out
==> /var/log/ambari-metrics-collector/hbase-ams-master-hdp4000.sre.iti.acceldata.dev.log <==
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 63374
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 32768
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 65536
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
2023-04-02 23:39:42,320 INFO [main] master.HMaster: STARTING service HMaster
2023-04-02 23:39:42,322 INFO [main] util.VersionInfo: HBase 2.0.2.3.1.4.0-313
2023-04-02 23:39:42,322 INFO [main] util.VersionInfo: Source code repository git://ctr-e139-1542663976389-113618-01-000005.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos7/SOURCES/hbase revision=
2023-04-02 23:39:42,322 INFO [main] util.VersionInfo: Compiled by jenkins on Wed Aug 21 16:27:48 UTC 2019
2023-04-02 23:39:42,322 INFO [main] util.VersionInfo: From source with checksum e335cc1e30039854f76e380656b67017
2023-04-02 23:39:42,724 INFO [main] master.HMasterCommandLine: Starting a zookeeper cluster
2023-04-02 23:39:42,745 INFO [main] server.ZooKeeperServer: Server environment:zookeeper.version=3.4.6-313--1, built on 08/21/2019 15:47 GMT
2023-04-02 23:39:42,745 INFO [main] server.ZooKeeperServer: Server environment:host.name=hdp4000.sre.iti.acceldata.dev
2023-04-02 23:39:42,745 INFO [main] server.ZooKeeperServer: Server environment:java.version=1.8.0_112
2023-04-02 23:39:42,745 INFO [main] server.ZooKeeperServer: Server environment:java.vendor=Oracle Corporation
2023-04-02 23:39:42,745 INFO [main] server.ZooKeeperServer: Server environment:java.home=/usr/jdk64/jdk1.8.0_112/jre
2023-04-02 23:39:42,745 INFO [main] server.ZooKeeperServer: Server environment:java.class.path=/etc/ams-hbase/conf:/usr/jdk64/jdk1.8.0_112/lib/tools.jar:/usr/lib/ams-hbase/:/usr/lib/ams-hbase//lib/animal-sniffer-annotations-1.17.jar:/usr/lib/ams-hbase//lib/antlr-runtime-3.5.2.jar:/usr/lib/ams-hbase//lib/aopalliance-1.0.jar:/usr/lib/ams-hbase//lib/aopalliance-repackaged-2.5.0-b32.jar:/usr/lib/ams-hbase//lib/audience-annotations-0.5.0.jar:/usr/lib/ams-hbase//lib/avro-1.7.7.jar:/usr/lib/ams-hbase//lib/aws-java-sdk-bundle-1.11.375.jar:/usr/lib/ams-hbase//lib/checker-qual-2.8.1.jar:/usr/lib/ams-hbase//lib/commons-beanutils-1.9.3.jar:/usr/lib/ams-hbase//lib/commons-cli-1.2.jar:/usr/lib/ams-hbase//lib/commons-codec-1.10.jar:/usr/lib/ams-hbase//lib/commons-collections-3.2.2.jar:/usr/lib/ams-hbase//lib/commons-compress-1.4.1.jar:/usr/lib/ams-hbase//lib/commons-configuration2-2.1.1.jar:/usr/lib/ams-hbase//lib/commons-crypto-1.0.0.jar:/usr/lib/ams-hbase//lib/commons-csv-1.0.jar:/usr/lib/ams-hbase//lib/commons-daemon-1.0.13.jar:/usr/lib/ams-hbase//lib/commons-io-2.5.jar:/usr/lib/ams-hbase//lib/commons-lang-2.6.jar:/usr/lib/ams-hbase//lib/commons-lang3-3.6.jar:/usr/lib/ams-hbase//lib/commons-logging-1.2.jar:/usr/lib/ams-hbase//lib/commons-math3-3.6.1.jar:/usr/lib/ams-hbase//lib/commons-net-3.6.jar:/usr/lib/ams-hbase//lib/curator-client-4.0.0.jar:/usr/lib/ams-hbase//lib/curator-framework-4.0.0.jar:/usr/lib/ams-hbase//lib/curator-recipes-4.0.0.jar:/usr/lib/ams-hbase//lib/disruptor-3.3.6.jar:/usr/lib/ams-hbase//lib/dnsjava-2.1.7.jar:/usr/lib/ams-hbase//lib/ehcache-3.3.1.jar:/usr/lib/ams-hbase//lib/error_prone_annotations-2.3.2.jar:/usr/lib/ams-hbase//lib/failureaccess-1.0.1.jar:/usr/lib/ams-hbase//lib/fst-2.50.jar:/usr/lib/ams-hbase//lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/usr/lib/ams-hbase//lib/gson-2.2.4.jar:/usr/lib/ams-hbase//lib/guava-28.0-jre.jar:/usr/lib/ams-hbase//lib/guice-4.0.jar:/usr/lib/ams-hbase//lib/guice-servlet-4.0.jar:/usr/lib/ams-hbase//lib/hadoop-annotations-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-auth-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-aws-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-client-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-common-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-distcp-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-hdfs-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-hdfs-client-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-mapreduce-client-app-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-mapreduce-client-common-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-mapreduce-client-core-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-mapreduce-client-hs-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-mapreduce-client-jobclient-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-mapreduce-client-shuffle-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-minicluster-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-api-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-client-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-common-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-registry-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-server-applicationhistoryservice-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-server-common-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-server-nodemanager-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-server-resourcemanager-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-server-timelineservice-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hadoop-yarn-server-web-proxy-3.1.1.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hamcrest-core-1.3.jar:/usr/lib/ams-hbase//lib/hbase-annotations-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-backup-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-client-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-common-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-endpoint-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-examples-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-external-blockcache-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-hadoop2-compat-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-hadoop-compat-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-http-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-it-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-mapreduce-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-metrics-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-metrics-api-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-procedure-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-protocol-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-protocol-shaded-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-replication-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-resource-bundle-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-rest-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-rsgroup-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-server-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-shaded-client-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-shaded-mapreduce-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-shaded-miscellaneous-2.2.0.jar:/usr/lib/ams-hbase//lib/hbase-shaded-netty-2.2.0.jar:/usr/lib/ams-hbase//lib/hbase-shaded-protobuf-2.2.0.jar:/usr/lib/ams-hbase//lib/hbase-shell-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-spark-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-testing-util-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-thrift-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/hbase-zookeeper-2.0.2.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/HikariCP-java7-2.4.12.jar:/usr/lib/ams-hbase//lib/hk2-api-2.5.0-b32.jar:/usr/lib/ams-hbase//lib/hk2-locator-2.5.0-b32.jar:/usr/lib/ams-hbase//lib/hk2-utils-2.5.0-b32.jar:/usr/lib/ams-hbase//lib/htrace-core-3.2.0-incubating.jar:/usr/lib/ams-hbase//lib/htrace-core4-4.2.0-incubating.jar:/usr/lib/ams-hbase//lib/httpclient-4.5.3.jar:/usr/lib/ams-hbase//lib/httpcore-4.4.6.jar:/usr/lib/ams-hbase//lib/j2objc-annotations-1.3.jar:/usr/lib/ams-hbase//lib/jackson-annotations-2.9.9.jar:/usr/lib/ams-hbase//lib/jackson-core-2.9.9.jar:/usr/lib/ams-hbase//lib/jackson-core-asl-1.9.13.jar:/usr/lib/ams-hbase//lib/jackson-databind-2.9.9.1.jar:/usr/lib/ams-hbase//lib/jackson-jaxrs-1.9.2.jar:/usr/lib/ams-hbase//lib/jackson-jaxrs-base-2.9.9.jar:/usr/lib/ams-hbase//lib/jackson-jaxrs-json-provider-2.9.9.jar:/usr/lib/ams-hbase//lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/ams-hbase//lib/jackson-module-jaxb-annotations-2.9.9.jar:/usr/lib/ams-hbase//lib/jackson-module-paranamer-2.9.9.jar:/usr/lib/ams-hbase//lib/jackson-module-scala_2.11-2.9.9.jar:/usr/lib/ams-hbase//lib/jackson-xc-1.9.2.jar:/usr/lib/ams-hbase//lib/jamon-runtime-2.4.1.jar:/usr/lib/ams-hbase//lib/javassist-3.20.0-GA.jar:/usr/lib/ams-hbase//lib/java-util-1.9.0.jar:/usr/lib/ams-hbase//lib/javax.annotation-api-1.2.jar:/usr/lib/ams-hbase//lib/javax.el-3.0.1-b08.jar:/usr/lib/ams-hbase//lib/javax.inject-2.5.0-b32.jar:/usr/lib/ams-hbase//lib/javax.servlet-api-3.1.0.jar:/usr/lib/ams-hbase//lib/javax.servlet.jsp-2.3.2.jar:/usr/lib/ams-hbase//lib/javax.servlet.jsp-api-2.3.1.jar:/usr/lib/ams-hbase//lib/javax.servlet.jsp.jstl-1.2.0.v201105211821.jar:/usr/lib/ams-hbase//lib/javax.servlet.jsp.jstl-1.2.2.jar:/usr/lib/ams-hbase//lib/javax.ws.rs-api-2.0.1.jar:/usr/lib/ams-hbase//lib/jaxb-api-2.2.12.jar:/usr/lib/ams-hbase//lib/jaxb-impl-2.2.3-1.jar:/usr/lib/ams-hbase//lib/jcip-annotations-1.0-1.jar:/usr/lib/ams-hbase//lib/jcodings-1.0.18.jar:/usr/lib/ams-hbase//lib/jersey-client-2.25.1.jar:/usr/lib/ams-hbase//lib/jersey-common-2.25.1.jar:/usr/lib/ams-hbase//lib/jersey-container-servlet-core-2.25.1.jar:/usr/lib/ams-hbase//lib/jersey-guava-2.25.1.jar:/usr/lib/ams-hbase//lib/jersey-media-jaxb-2.25.1.jar:/usr/lib/ams-hbase//lib/jersey-server-2.25.1.jar:/usr/lib/ams-hbase//lib/jettison-1.3.8.jar:/usr/lib/ams-hbase//lib/jetty-http-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-io-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-jmx-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-jsp-9.2.28.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-schemas-3.1.M0.jar:/usr/lib/ams-hbase//lib/jetty-security-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-server-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-servlet-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-util-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-util-ajax-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-webapp-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/jetty-xml-9.3.27.v20190418.jar:/usr/lib/ams-hbase//lib/joni-2.1.11.jar:/usr/lib/ams-hbase//lib/jsch-0.1.54.jar:/usr/lib/ams-hbase//lib/json-io-2.5.1.jar:/usr/lib/ams-hbase//lib/jsr311-api-1.1.1.jar:/usr/lib/ams-hbase//lib/junit-4.12.jar:/usr/lib/ams-hbase//lib/leveldbjni-all-1.8.jar:/usr/lib/ams-hbase//lib/libthrift-0.9.3-1.jar:/usr/lib/ams-hbase//lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/usr/lib/ams-hbase//lib/log4j-1.2.17.jar:/usr/lib/ams-hbase//lib/metrics-core-3.2.1.jar:/usr/lib/ams-hbase//lib/mssql-jdbc-6.2.1.jre7.jar:/usr/lib/ams-hbase//lib/netty-all-4.0.52.Final.jar:/usr/lib/ams-hbase//lib/nimbus-jose-jwt-4.41.1.jar:/usr/lib/ams-hbase//lib/objenesis-2.1.jar:/usr/lib/ams-hbase//lib/okhttp-2.7.5.jar:/usr/lib/ams-hbase//lib/okio-1.6.0.jar:/usr/lib/ams-hbase//lib/org.eclipse.jdt.core-3.8.2.v20130121.jar:/usr/lib/ams-hbase//lib/osgi-resource-locator-1.0.1.jar:/usr/lib/ams-hbase//lib/paranamer-2.3.jar:/usr/lib/ams-hbase//lib/phoenix-5.0.0.3.1.4.0-313-server.jar:/usr/lib/ams-hbase//lib/phoenix-core-5.0.0.3.1.4.0-313.jar:/usr/lib/ams-hbase//lib/protobuf-java-2.5.0.jar:/usr/lib/ams-hbase//lib/re2j-1.1.jar:/usr/lib/ams-hbase//lib/slf4j-api-1.7.25.jar:/usr/lib/ams-hbase//lib/slf4j-log4j12-1.7.25.jar:/usr/lib/ams-hbase//lib/snappy-java-1.0.5.jar:/usr/lib/ams-hbase//lib/spymemcached-2.12.2.jar:/usr/lib/ams-hbase//lib/stax2-api-3.1.4.jar:/usr/lib/ams-hbase//lib/validation-api-1.1.0.Final.jar:/usr/lib/ams-hbase//lib/woodstox-core-5.0.3.jar:/usr/lib/ams-hbase//lib/xz-1.0.jar:/usr/lib/ams-hbase//lib/zookeeper-3.4.6.3.1.4.0-313.jar:
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:java.library.path=/usr/lib/ams-hbase/lib/hadoop-native/
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:java.io.tmpdir=/var/lib/ambari-metrics-collector/hbase-tmp
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:java.compiler=<NA>
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:os.name=Linux
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:os.arch=amd64
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:os.version=3.10.0-1160.49.1.el7.x86_64
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:user.name=ams
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:user.home=/home/ams
2023-04-02 23:39:42,746 INFO [main] server.ZooKeeperServer: Server environment:user.dir=/home/ams
2023-04-02 23:39:42,771 INFO [main] server.ZooKeeperServer: Created server with tickTime 6000 minSessionTimeout 12000 maxSessionTimeout 120000 datadir /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/zookeeper_0/version-2 snapdir /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/zookeeper_0/version-2
2023-04-02 23:39:42,771 INFO [main] server.ZooKeeperServer: minSessionTimeout set to -1
2023-04-02 23:39:42,771 INFO [main] server.ZooKeeperServer: maxSessionTimeout set to -1
2023-04-02 23:39:42,789 INFO [main] server.NIOServerCnxnFactory: binding to port 0.0.0.0/0.0.0.0:61181
2023-04-02 23:39:42,938 ERROR [main] server.ZooKeeperServer: ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
==> /var/log/ambari-metrics-collector/gc.log-202301231409 <==
Java HotSpot(TM) 64-Bit Server VM (25.112-b15) for linux-amd64 JRE (1.8.0_112-b15), built on Sep 22 2016 21:10:53 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
Memory: 4k page, physical 16247588k(2798184k free), swap 1679356k(21660k free)
CommandLine flags: -XX:CMSInitiatingOccupancyFraction=70 -XX:ErrorFile=/var/log/ambari-metrics-collector/hs_err_pid%p.log -XX:InitialHeapSize=1342177280 -XX:MaxHeapSize=1342177280 -XX:MaxNewSize=201326592 -XX:MaxTenuringThreshold=6 -XX:NewSize=201326592 -XX:OldPLABSize=16 -XX:OnOutOfMemoryError=kill -9 %p -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
Heap
par new generation total 176960K, used 69221K [0x00000000b0000000, 0x00000000bc000000, 0x00000000bc000000)
eden space 157312K, 44% used [0x00000000b0000000, 0x00000000b4399750, 0x00000000b99a0000)
from space 19648K, 0% used [0x00000000b99a0000, 0x00000000b99a0000, 0x00000000bacd0000)
to space 19648K, 0% used [0x00000000bacd0000, 0x00000000bacd0000, 0x00000000bc000000)
concurrent mark-sweep generation total 1114112K, used 0K [0x00000000bc000000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 10313K, capacity 10502K, committed 10752K, reserved 1058816K
class space used 1132K, capacity 1220K, committed 1280K, reserved 1048576K
==> /var/log/ambari-metrics-collector/SecurityAuth.audit <==
2023-01-29 15:05:12,533 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:45288, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:10:12,962 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:45988, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:15:12,531 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:46668, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:20:12,717 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:47372, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:25:12,644 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:48048, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:30:12,532 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:48744, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:35:16,542 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:49434, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:40:12,547 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:50100, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:45:12,533 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:50784, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:50:12,541 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:51482, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 15:55:13,893 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:52176, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:00:12,529 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:52868, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:05:12,582 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:53568, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:10:13,160 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:54258, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:15:12,532 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:54938, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:20:12,830 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:55632, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:25:16,445 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:56306, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:30:12,536 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:56992, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:35:12,553 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:57660, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:40:13,094 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:58344, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:45:12,592 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:59030, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:50:12,537 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:59728, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 16:55:12,900 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:60416, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:00:12,545 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:32884, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:05:14,680 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:33544, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:10:12,681 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:34218, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:15:12,566 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:34902, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:20:13,475 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:35600, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:25:16,401 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:36268, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:30:12,598 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:36962, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:35:12,529 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:37650, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:40:12,543 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:38342, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:45:17,002 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:39022, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:50:12,724 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:39702, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 17:55:12,533 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:40390, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 18:00:12,537 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:41070, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-01-29 18:05:12,730 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 10.90.6.71:41730, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
2023-04-02 23:36:24,300 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 127.0.0.1:47896, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=RegionServerStatusService
2023-04-02 23:36:28,720 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 127.0.0.1:32954, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=AdminService
2023-04-02 23:36:29,833 INFO SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 127.0.0.1:32956, version=2.0.2.3.1.4.0-313, sasl=false, ugi=ams (auth:SIMPLE), service=ClientService
==> /var/log/ambari-metrics-collector/ambari-metrics-collector.out <==
Error: Could not find or load main class org.apache.ambari.metrics.AMSApplicationServer
==> /var/log/ambari-metrics-collector/collector-gc.log-202301231409 <==
Java HotSpot(TM) 64-Bit Server VM (25.112-b15) for linux-amd64 JRE (1.8.0_112-b15), built on Sep 22 2016 21:10:53 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
Memory: 4k page, physical 16247588k(11233300k free), swap 1679356k(1679356k free)
CommandLine flags: -XX:InitialHeapSize=536870912 -XX:MaxHeapSize=536870912 -XX:MaxNewSize=178958336 -XX:MaxTenuringThreshold=6 -XX:NewSize=178958336 -XX:OldPLABSize=16 -XX:OldSize=357912576 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
Heap
par new generation total 157248K, used 8386K [0x00000000e0000000, 0x00000000eaaa0000, 0x00000000eaaa0000)
eden space 139776K, 6% used [0x00000000e0000000, 0x00000000e0830a70, 0x00000000e8880000)
from space 17472K, 0% used [0x00000000e8880000, 0x00000000e8880000, 0x00000000e9990000)
to space 17472K, 0% used [0x00000000e9990000, 0x00000000e9990000, 0x00000000eaaa0000)
concurrent mark-sweep generation total 349568K, used 0K [0x00000000eaaa0000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 2530K, capacity 4480K, committed 4480K, reserved 1056768K
class space used 283K, capacity 384K, committed 384K, reserved 1048576K
==> /var/log/ambari-metrics-collector/gc.log-202304022335 <==
Java HotSpot(TM) 64-Bit Server VM (25.112-b15) for linux-amd64 JRE (1.8.0_112-b15), built on Sep 22 2016 21:10:53 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
Memory: 4k page, physical 16247588k(7102812k free), swap 1679356k(1180k free)
CommandLine flags: -XX:CMSInitiatingOccupancyFraction=70 -XX:ErrorFile=/var/log/ambari-metrics-collector/hs_err_pid%p.log -XX:InitialHeapSize=1610612736 -XX:MaxHeapSize=1610612736 -XX:MaxNewSize=268435456 -XX:MaxTenuringThreshold=6 -XX:NewSize=268435456 -XX:OldPLABSize=16 -XX:OnOutOfMemoryError=kill -9 %p -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
2023-04-02T23:36:19.022+0530: 27.284: [GC (Allocation Failure) 2023-04-02T23:36:19.022+0530: 27.285: [ParNew: 209792K->16008K(235968K), 0.0654369 secs] 209792K->16008K(1546688K), 0.0657816 secs] [Times: user=0.17 sys=0.06, real=0.06 secs]
2023-04-02T23:36:21.089+0530: 29.352: [GC (CMS Initial Mark) [1 CMS-initial-mark: 0K(1310720K)] 153748K(1546688K), 0.0105956 secs] [Times: user=0.10 sys=0.00, real=0.02 secs]
2023-04-02T23:36:21.100+0530: 29.363: [CMS-concurrent-mark-start]
2023-04-02T23:36:21.101+0530: 29.364: [CMS-concurrent-mark: 0.001/0.001 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
2023-04-02T23:36:21.101+0530: 29.364: [CMS-concurrent-preclean-start]
2023-04-02T23:36:21.106+0530: 29.369: [CMS-concurrent-preclean: 0.005/0.005 secs] [Times: user=0.03 sys=0.01, real=0.00 secs]
2023-04-02T23:36:21.106+0530: 29.369: [CMS-concurrent-abortable-preclean-start]
2023-04-02T23:36:22.524+0530: 30.787: [GC (Allocation Failure) 2023-04-02T23:36:22.524+0530: 30.787: [ParNew: 225800K->24167K(235968K), 0.1943044 secs] 225800K->30289K(1546688K), 0.1946002 secs] [Times: user=0.42 sys=0.22, real=0.19 secs]
2023-04-02T23:36:24.554+0530: 32.817: [CMS-concurrent-abortable-preclean: 2.384/3.448 secs] [Times: user=9.35 sys=1.01, real=3.45 secs]
2023-04-02T23:36:24.555+0530: 32.818: [GC (CMS Final Remark) [YG occupancy: 129798 K (235968 K)]2023-04-02T23:36:24.555+0530: 32.818: [Rescan (parallel) , 0.0095596 secs]2023-04-02T23:36:24.565+0530: 32.828: [weak refs processing, 0.0000689 secs]2023-04-02T23:36:24.565+0530: 32.828: [class unloading, 0.0123779 secs]2023-04-02T23:36:24.578+0530: 32.840: [scrub symbol table, 0.0115901 secs]2023-04-02T23:36:24.589+0530: 32.852: [scrub string table, 0.0012807 secs][1 CMS-remark: 6121K(1310720K)] 135920K(1546688K), 0.0364442 secs] [Times: user=0.13 sys=0.00, real=0.04 secs]
2023-04-02T23:36:24.592+0530: 32.855: [CMS-concurrent-sweep-start]
2023-04-02T23:36:24.601+0530: 32.863: [CMS-concurrent-sweep: 0.007/0.008 secs] [Times: user=0.04 sys=0.01, real=0.01 secs]
2023-04-02T23:36:24.601+0530: 32.864: [CMS-concurrent-reset-start]
2023-04-02T23:36:24.660+0530: 32.923: [CMS-concurrent-reset: 0.054/0.059 secs] [Times: user=0.11 sys=0.07, real=0.06 secs]
2023-04-02T23:36:29.159+0530: 37.422: [GC (Allocation Failure) 2023-04-02T23:36:29.160+0530: 37.422: [ParNew: 233959K->23749K(235968K), 0.0625500 secs] 240081K->35746K(1546688K), 0.0631054 secs] [Times: user=0.18 sys=0.05, real=0.06 secs]
2023-04-02T23:38:35.101+0530: 163.364: [GC (Allocation Failure) 2023-04-02T23:38:35.101+0530: 163.364: [ParNew: 233541K->18154K(235968K), 0.0370252 secs] 245538K->37068K(1546688K), 0.0375213 secs] [Times: user=0.17 sys=0.02, real=0.03 secs]
Heap
par new generation total 235968K, used 110735K [0x00000000a0000000, 0x00000000b0000000, 0x00000000b0000000)
eden space 209792K, 44% used [0x00000000a0000000, 0x00000000a5a69610, 0x00000000acce0000)
from space 26176K, 69% used [0x00000000acce0000, 0x00000000ade9a9e8, 0x00000000ae670000)
to space 26176K, 0% used [0x00000000ae670000, 0x00000000ae670000, 0x00000000b0000000)
concurrent mark-sweep generation total 1310720K, used 18913K [0x00000000b0000000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 62664K, capacity 63532K, committed 63692K, reserved 1105920K
class space used 7116K, capacity 7312K, committed 7400K, reserved 1048576K
==> /var/log/ambari-metrics-collector/collector-gc.log-202304022335 <==
Java HotSpot(TM) 64-Bit Server VM (25.112-b15) for linux-amd64 JRE (1.8.0_112-b15), built on Sep 22 2016 21:10:53 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
Memory: 4k page, physical 16247588k(7028496k free), swap 1679356k(1180k free)
CommandLine flags: -XX:InitialHeapSize=536870912 -XX:MaxHeapSize=536870912 -XX:MaxNewSize=178958336 -XX:MaxTenuringThreshold=6 -XX:NewSize=178958336 -XX:OldPLABSize=16 -XX:OldSize=357912576 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
Heap
par new generation total 157248K, used 8386K [0x00000000e0000000, 0x00000000eaaa0000, 0x00000000eaaa0000)
eden space 139776K, 6% used [0x00000000e0000000, 0x00000000e0830a70, 0x00000000e8880000)
from space 17472K, 0% used [0x00000000e8880000, 0x00000000e8880000, 0x00000000e9990000)
to space 17472K, 0% used [0x00000000e9990000, 0x00000000e9990000, 0x00000000eaaa0000)
concurrent mark-sweep generation total 349568K, used 0K [0x00000000eaaa0000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 2530K, capacity 4480K, committed 4480K, reserved 1056768K
class space used 283K, capacity 384K, committed 384K, reserved 1048576K
==> /var/log/ambari-metrics-collector/hbase-ams-master-hdp4000.sre.iti.acceldata.dev.out.3 <==
/usr/lib/ams-hbase/bin/hbase-daemon.sh: line 250: 95198 Killed nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase --config "${HBASE_CONF_DIR}" $command "$@" start >> ${HBASE_LOGOUT} 2>&1
binding in [jar:file:/usr/lib/ams-hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
==> /var/log/ambari-metrics-collector/hbase-ams-master-hdp4000.sre.iti.acceldata.dev.out.2 <==
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/phoenix-5.0.0.3.1.4.0-313-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
==> /var/log/ambari-metrics-collector/hbase-ams-master-hdp4000.sre.iti.acceldata.dev.out.1 <==
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/phoenix-5.0.0.3.1.4.0-313-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
==> /var/log/ambari-metrics-collector/hbase-ams-master-hdp4000.sre.iti.acceldata.dev.out <==
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/phoenix-5.0.0.3.1.4.0-313-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/ams-hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
==> /var/log/ambari-metrics-collector/gc.log-202304022339 <==
==> /var/log/ambari-metrics-collector/collector-gc.log-202304022339 <==
Java HotSpot(TM) 64-Bit Server VM (25.112-b15) for linux-amd64 JRE (1.8.0_112-b15), built on Sep 22 2016 21:10:53 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
Memory: 4k page, physical 16247588k(6973952k free), swap 1679356k(1184k free)
CommandLine flags: -XX:InitialHeapSize=536870912 -XX:MaxHeapSize=536870912 -XX:MaxNewSize=178958336 -XX:MaxTenuringThreshold=6 -XX:NewSize=178958336 -XX:OldPLABSize=16 -XX:OldSize=357912576 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
Heap
par new generation total 157248K, used 8386K [0x00000000e0000000, 0x00000000eaaa0000, 0x00000000eaaa0000)
eden space 139776K, 6% used [0x00000000e0000000, 0x00000000e0830a70, 0x00000000e8880000)
from space 17472K, 0% used [0x00000000e8880000, 0x00000000e8880000, 0x00000000e9990000)
to space 17472K, 0% used [0x00000000e9990000, 0x00000000e9990000, 0x00000000eaaa0000)
concurrent mark-sweep generation total 349568K, used 0K [0x00000000eaaa0000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 2530K, capacity 4480K, committed 4480K, reserved 1056768K
class space used 283K, capacity 384K, committed 384K, reserved 1048576K
2023-04-02 23:39:44,896 - Skipping stack-select on AMBARI_METRICS because it does not exist in the stack-select package structure.

Command failed after 1 tries

 

 

3,495 Views
0 Kudos
2 ACCEPTED SOLUTIONS

avatar
author-rank akhilsnaik author-rank
Guru

Created ‎04-04-2023 11:48 PM 

Error: Could not find or load main class org.apache.ambari.metrics.AMSApplicationServer

 

From the Error it looks like you have some issue with AMS package . have you done anything on the node recently.

you can either try reinstall button in the Ambari metrics collector host component page or try command : 

yum reinstall ambari-metrics-collector

and see if that helps.

 

View solution in original post

3,451 Views
0 Kudos

avatar
author-rank Shelton
Master Mentor

Created ‎04-06-2023 12:49 PM

@SSandhu 
The first question is I am just wondering if you have an HDP subscription. If not sure you could independently run the below to see whether the repo URL is valid  and reachable
Display enabled  HDP software repositories

 

Spoiler
# yum repolist

Clean out all HDP packages and metadata from cache

 

 

Spoiler
# yum clean all

Refresh HDP packages on your system

 

 

Spoiler
# yum update

Reinstall AMC

Spoiler
yum reinstall ambari-metrics-collector

The above steps should help you resolve  AMC does not exist in the stack-select package 

 




 

View solution in original post

3,423 Views
0 Kudos
3 REPLIES 3

avatar
author-rank VidyaSargur author-rank
Community Manager

Created ‎04-03-2023 08:11 AM

@SSandhu, Welcome to our community! To help you get the best possible answer, I have tagged our Ambari experts  @akhilsnaik @Asok @Scharan @pvishnu  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.


Regards,

Vidya Sargur,
Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
3,493 Views
0 Kudos

avatar
author-rank akhilsnaik author-rank
Guru

Created ‎04-04-2023 11:48 PM 

Error: Could not find or load main class org.apache.ambari.metrics.AMSApplicationServer

 

From the Error it looks like you have some issue with AMS package . have you done anything on the node recently.

you can either try reinstall button in the Ambari metrics collector host component page or try command : 

yum reinstall ambari-metrics-collector

and see if that helps.

 

3,452 Views
0 Kudos

avatar
author-rank Shelton
Master Mentor

Created ‎04-06-2023 12:49 PM

@SSandhu 
The first question is I am just wondering if you have an HDP subscription. If not sure you could independently run the below to see whether the repo URL is valid  and reachable
Display enabled  HDP software repositories

 

Spoiler
# yum repolist

Clean out all HDP packages and metadata from cache

 

 

Spoiler
# yum clean all

Refresh HDP packages on your system

 

 

Spoiler
# yum update

Reinstall AMC

Spoiler
yum reinstall ambari-metrics-collector

The above steps should help you resolve  AMC does not exist in the stack-select package 

 




 

3,424 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements