Hi,
I just upgraded Ambari from 2.1 to 2.2, my HDP version is 2.1.0, all on CentOS 6.7. I have a new strange issue with starting of MapReduce2 and YARN services. All is ending with this message:
stderr: /var/lib/ambari-agent/data/errors-4003.txt
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 221, in <module>
Resourcemanager().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 110, in start
import params
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params.py", line 28, in <module>
from params_linux import *
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py", line 153, in <module>
ats_leveldb_lock_file = os.path.join(ats_leveldb_dir, "leveldb-timeline-store.ldb", "LOCK")
File "/usr/lib64/python2.6/posixpath.py", line 67, in join
elif path == '' or path.endswith('/'):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 81, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'yarn.timeline-service.leveldb-timeline-store.path' was not found in configurations dictionary!stdout: /var/lib/ambari-agent/data/output-4003.txt
2016-01-21 14:05:32,174 - Using hadoop conf dir: /etc/hadoop/conf
2016-01-21 14:05:32,282 - Using hadoop conf dir: /etc/hadoop/conf
2016-01-21 14:05:32,284 - Group['hadoop'] {}
2016-01-21 14:05:32,286 - Group['users'] {}
2016-01-21 14:05:32,286 - User['mapred'] {'gid': 'hadoop', 'groups': ['hadoop']}
2016-01-21 14:05:32,288 - User['ambari-qa'] {'gid': 'hadoop', 'groups': ['users']}
2016-01-21 14:05:32,289 - User['zookeeper'] {'gid': 'hadoop', 'groups': ['hadoop']}
2016-01-21 14:05:32,290 - User['hdfs'] {'gid': 'hadoop', 'groups': ['hadoop']}
2016-01-21 14:05:32,291 - User['yarn'] {'gid': 'hadoop', 'groups': ['hadoop']}
2016-01-21 14:05:32,292 - User['ams'] {'gid': 'hadoop', 'groups': ['hadoop']}
2016-01-21 14:05:32,292 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-01-21 14:05:32,294 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2016-01-21 14:05:32,343 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2016-01-21 14:05:32,344 - User['hdfs'] {'ignore_failures': False}
2016-01-21 14:05:32,345 - User['hdfs'] {'ignore_failures': False, 'groups': ['hadoop']}
2016-01-21 14:05:32,345 - Directory['/etc/hadoop'] {'mode': 0755}
2016-01-21 14:05:32,346 - Directory['/etc/hadoop/conf.empty'] {'owner': 'root', 'group': 'hadoop', 'recursive': True}
2016-01-21 14:05:32,346 - Link['/etc/hadoop/conf'] {'not_if': 'ls /etc/hadoop/conf', 'to': '/etc/hadoop/conf.empty'}
2016-01-21 14:05:32,393 - Skipping Link['/etc/hadoop/conf'] due to not_if
2016-01-21 14:05:32,407 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2016-01-21 14:05:32,408 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777}
2016-01-21 14:05:32,423 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2016-01-21 14:05:32,477 - Skipping Execute[('setenforce', '0')] due to not_if
2016-01-21 14:05:32,477 - Directory['/var/log/hadoop'] {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2016-01-21 14:05:32,480 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'}
2016-01-21 14:05:32,480 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'}
2016-01-21 14:05:32,481 - File['/var/lib/ambari-agent/lib/fast-hdfs-resource.jar'] {'content': StaticFile('fast-hdfs-resource.jar'), 'mode': 0644}
2016-01-21 14:05:32,538 - File['/etc/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2016-01-21 14:05:32,540 - File['/etc/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2016-01-21 14:05:32,541 - File['/etc/hadoop/conf/log4j.properties'] {'content': ..., 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2016-01-21 14:05:32,549 - File['/etc/hadoop/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2016-01-21 14:05:32,549 - File['/etc/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2016-01-21 14:05:32,550 - File['/etc/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2016-01-21 14:05:32,554 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2016-01-21 14:05:32,602 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2016-01-21 14:05:32,845 - Using hadoop conf dir: /etc/hadoop/conf
2016-01-21 14:05:32,845 - Skipping get_hdp_version since hdp-select is not yet available
2016-01-21 14:05:32,846 - Using hadoop conf dir: /etc/hadoop/confCan you please recommend what shall I do? Thank you so much.
Regards,
Pavel
