2016-04-21 12:19:45,441 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:45,511 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:45,581 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:45,648 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:45,650 - In the middle of a stack upgrade/downgrade for Stack HDP and destination version 2.3.4.0-3485, determining which hadoop conf dir to use. 2016-04-21 12:19:45,650 - Hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:45,651 - The hadoop conf dir /usr/hdp/2.3.4.0-3485/hadoop/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-04-21 12:19:45,651 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-04-21 12:19:45,651 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-04-21 12:19:45,719 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-04-21 12:19:45,720 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-04-21 12:19:45,783 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-04-21 12:19:45,783 - Ensuring that hadoop has the correct symlink structure 2016-04-21 12:19:45,783 - Using hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:45,907 - In the middle of a stack upgrade/downgrade for Stack HDP and destination version 2.3.4.0-3485, determining which hadoop conf dir to use. 2016-04-21 12:19:45,907 - Hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:45,907 - The hadoop conf dir /usr/hdp/2.3.4.0-3485/hadoop/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-04-21 12:19:45,907 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-04-21 12:19:45,907 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-04-21 12:19:45,969 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-04-21 12:19:45,970 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-04-21 12:19:46,038 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-04-21 12:19:46,039 - Ensuring that hadoop has the correct symlink structure 2016-04-21 12:19:46,039 - Using hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:46,040 - Group['hadoop'] {} 2016-04-21 12:19:46,042 - Group['users'] {} 2016-04-21 12:19:46,042 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']} 2016-04-21 12:19:46,045 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']} 2016-04-21 12:19:46,047 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']} 2016-04-21 12:19:46,049 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']} 2016-04-21 12:19:46,051 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']} 2016-04-21 12:19:46,053 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']} 2016-04-21 12:19:46,055 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']} 2016-04-21 12:19:46,057 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2016-04-21 12:19:46,061 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'} 2016-04-21 12:19:46,097 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if 2016-04-21 12:19:46,098 - Group['hdfs'] {} 2016-04-21 12:19:46,098 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'hdfs']} 2016-04-21 12:19:46,100 - Directory['/etc/hadoop'] {'mode': 0755} 2016-04-21 12:19:46,122 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2016-04-21 12:19:46,122 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777} 2016-04-21 12:19:46,137 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'} 2016-04-21 12:19:46,176 - Skipping Execute[('setenforce', '0')] due to not_if 2016-04-21 12:19:46,177 - Directory['/var/log/hadoop'] {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:46,182 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:46,183 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:46,190 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'} 2016-04-21 12:19:46,194 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'} 2016-04-21 12:19:46,195 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/log4j.properties'] {'content': ..., 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644} 2016-04-21 12:19:46,205 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'} 2016-04-21 12:19:46,205 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755} 2016-04-21 12:19:46,207 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'} 2016-04-21 12:19:46,211 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'} 2016-04-21 12:19:46,249 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755} 2016-04-21 12:19:46,514 - In the middle of a stack upgrade/downgrade for Stack HDP and destination version 2.3.4.0-3485, determining which hadoop conf dir to use. 2016-04-21 12:19:46,581 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:46,582 - Hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:46,582 - The hadoop conf dir /usr/hdp/2.3.4.0-3485/hadoop/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-04-21 12:19:46,582 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-04-21 12:19:46,583 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-04-21 12:19:46,651 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-04-21 12:19:46,651 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-04-21 12:19:46,715 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-04-21 12:19:46,715 - Ensuring that hadoop has the correct symlink structure 2016-04-21 12:19:46,716 - Using hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:46,854 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:46,919 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:46,983 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:46,984 - In the middle of a stack upgrade/downgrade for Stack HDP and destination version 2.3.4.0-3485, determining which hadoop conf dir to use. 2016-04-21 12:19:47,048 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:47,048 - Hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:47,048 - The hadoop conf dir /usr/hdp/2.3.4.0-3485/hadoop/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-04-21 12:19:47,049 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-04-21 12:19:47,049 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-04-21 12:19:47,114 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-04-21 12:19:47,114 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-04-21 12:19:47,181 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-04-21 12:19:47,182 - Ensuring that hadoop has the correct symlink structure 2016-04-21 12:19:47,182 - Using hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:47,187 - Execute['export HADOOP_LIBEXEC_DIR=/usr/hdp/2.3.4.0-3485/hadoop/libexec && /usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh --config /usr/hdp/2.3.4.0-3485/hadoop/conf stop nodemanager'] {'user': 'yarn'} 2016-04-21 12:19:52,380 - Executing NodeManager Stack Upgrade pre-restart 2016-04-21 12:19:52,383 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-04-21 12:19:52,383 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-04-21 12:19:52,454 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-04-21 12:19:52,454 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-04-21 12:19:52,521 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-04-21 12:19:52,521 - Ensuring that hadoop has the correct symlink structure 2016-04-21 12:19:52,522 - Execute[('hdp-select', 'set', 'hadoop-yarn-nodemanager', '2.3.4.0-3485')] {'sudo': True} 2016-04-21 12:19:52,589 - In the middle of a stack upgrade/downgrade for Stack HDP and destination version 2.3.4.0-3485, determining which hadoop conf dir to use. 2016-04-21 12:19:52,652 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:52,652 - Hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:52,652 - The hadoop conf dir /usr/hdp/2.3.4.0-3485/hadoop/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-04-21 12:19:52,653 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-04-21 12:19:52,653 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-04-21 12:19:52,717 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-04-21 12:19:52,717 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-04-21 12:19:52,791 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-04-21 12:19:52,791 - Ensuring that hadoop has the correct symlink structure 2016-04-21 12:19:52,791 - Using hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:52,792 - After ('hdp-select', 'set', 'hadoop-yarn-nodemanager', '2.3.4.0-3485'), reloaded module status_params 2016-04-21 12:19:52,921 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:52,993 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:53,058 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:53,059 - In the middle of a stack upgrade/downgrade for Stack HDP and destination version 2.3.4.0-3485, determining which hadoop conf dir to use. 2016-04-21 12:19:53,132 - hadoop-yarn-nodemanager is currently at version 2.3.4.0-3485 2016-04-21 12:19:53,132 - Hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:53,132 - The hadoop conf dir /usr/hdp/2.3.4.0-3485/hadoop/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-04-21 12:19:53,132 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-04-21 12:19:53,133 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-04-21 12:19:53,200 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-04-21 12:19:53,201 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-04-21 12:19:53,265 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-04-21 12:19:53,265 - Ensuring that hadoop has the correct symlink structure 2016-04-21 12:19:53,265 - Using hadoop conf dir: /usr/hdp/2.3.4.0-3485/hadoop/conf 2016-04-21 12:19:53,268 - After ('hdp-select', 'set', 'hadoop-yarn-nodemanager', '2.3.4.0-3485'), reloaded module params_linux 2016-04-21 12:19:53,269 - After ('hdp-select', 'set', 'hadoop-yarn-nodemanager', '2.3.4.0-3485'), reloaded module params 2016-04-21 12:19:53,272 - Directory['/tmp/hadoop/yarn/local'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,273 - Changing permission for /tmp/hadoop/yarn/local from 755 to 775 2016-04-21 12:19:53,274 - Directory['/var/hadoop/yarn/local'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,274 - Changing permission for /var/hadoop/yarn/local from 755 to 775 2016-04-21 12:19:53,274 - Directory['/u02A/hadoop/yarn/local'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,274 - Changing permission for /u02A/hadoop/yarn/local from 755 to 775 2016-04-21 12:19:53,275 - Directory['/u02B/hadoop/yarn/local'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,275 - Changing permission for /u02B/hadoop/yarn/local from 755 to 775 2016-04-21 12:19:53,275 - Directory['/tmp/hadoop/yarn/log'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,276 - Directory['/var/hadoop/yarn/log'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,276 - Directory['/u02A/hadoop/yarn/log'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,277 - Directory['/u02B/hadoop/yarn/log'] {'group': 'hadoop', 'recursive': True, 'cd_access': 'a', 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2016-04-21 12:19:53,277 - Execute[('chmod', '-R', '755', '/tmp/hadoop/yarn/local', '/var/hadoop/yarn/local', '/u02A/hadoop/yarn/local', '/u02B/hadoop/yarn/local')] {'sudo': True} 2016-04-21 12:19:53,344 - Directory['/var/log/hadoop-yarn/nodemanager/recovery-state'] {'owner': 'yarn', 'mode': 0755, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,345 - Directory['/var/run/hadoop-yarn'] {'owner': 'yarn', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,345 - Directory['/var/run/hadoop-yarn/yarn'] {'owner': 'yarn', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,346 - Directory['/var/log/hadoop-yarn/yarn'] {'owner': 'yarn', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,346 - Directory['/var/run/hadoop-mapreduce'] {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,346 - Directory['/var/run/hadoop-mapreduce/mapred'] {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,347 - Directory['/var/log/hadoop-mapreduce'] {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,347 - Directory['/var/log/hadoop-mapreduce/mapred'] {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,348 - Directory['/var/log/hadoop-yarn'] {'owner': 'yarn', 'ignore_failures': True, 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,348 - XmlConfig['core-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hdfs', 'configurations': ...} 2016-04-21 12:19:53,357 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/core-site.xml 2016-04-21 12:19:53,358 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/core-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,375 - XmlConfig['hdfs-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hdfs', 'configurations': ...} 2016-04-21 12:19:53,383 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/hdfs-site.xml 2016-04-21 12:19:53,384 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/hdfs-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,426 - XmlConfig['mapred-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...} 2016-04-21 12:19:53,434 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/mapred-site.xml 2016-04-21 12:19:53,435 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/mapred-site.xml'] {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,471 - Changing owner for /usr/hdp/2.3.4.0-3485/hadoop/conf/mapred-site.xml from 39209 to yarn 2016-04-21 12:19:53,471 - XmlConfig['yarn-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...} 2016-04-21 12:19:53,480 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/yarn-site.xml 2016-04-21 12:19:53,480 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/yarn-site.xml'] {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,562 - XmlConfig['capacity-scheduler.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...} 2016-04-21 12:19:53,570 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/capacity-scheduler.xml 2016-04-21 12:19:53,570 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/capacity-scheduler.xml'] {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,581 - Changing owner for /usr/hdp/2.3.4.0-3485/hadoop/conf/capacity-scheduler.xml from 39207 to yarn 2016-04-21 12:19:53,582 - File['/etc/hadoop/conf/yarn.exclude'] {'owner': 'yarn', 'group': 'hadoop'} 2016-04-21 12:19:53,585 - File['/etc/security/limits.d/yarn.conf'] {'content': Template('yarn.conf.j2'), 'mode': 0644} 2016-04-21 12:19:53,587 - File['/etc/security/limits.d/mapreduce.conf'] {'content': Template('mapreduce.conf.j2'), 'mode': 0644} 2016-04-21 12:19:53,591 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/yarn-env.sh'] {'content': InlineTemplate(...), 'owner': 'yarn', 'group': 'hadoop', 'mode': 0755} 2016-04-21 12:19:53,592 - File['/usr/hdp/current/hadoop-yarn-nodemanager/bin/container-executor'] {'group': 'hadoop', 'mode': 02050} 2016-04-21 12:19:53,594 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/container-executor.cfg'] {'content': Template('container-executor.cfg.j2'), 'group': 'hadoop', 'mode': 0644} 2016-04-21 12:19:53,595 - Directory['/cgroups_test/cpu'] {'mode': 0755, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,596 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/mapred-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'mode': 0755} 2016-04-21 12:19:53,599 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/taskcontroller.cfg'] {'content': Template('taskcontroller.cfg.j2'), 'owner': 'hdfs'} 2016-04-21 12:19:53,599 - XmlConfig['mapred-site.xml'] {'owner': 'mapred', 'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'configuration_attributes': {}, 'configurations': ...} 2016-04-21 12:19:53,608 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/mapred-site.xml 2016-04-21 12:19:53,608 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/mapred-site.xml'] {'owner': 'mapred', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,643 - Changing owner for /usr/hdp/2.3.4.0-3485/hadoop/conf/mapred-site.xml from 39208 to mapred 2016-04-21 12:19:53,643 - XmlConfig['capacity-scheduler.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'configuration_attributes': {}, 'configurations': ...} 2016-04-21 12:19:53,651 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/capacity-scheduler.xml 2016-04-21 12:19:53,651 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/capacity-scheduler.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,663 - Changing owner for /usr/hdp/2.3.4.0-3485/hadoop/conf/capacity-scheduler.xml from 39208 to hdfs 2016-04-21 12:19:53,663 - XmlConfig['ssl-client.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'configuration_attributes': {}, 'configurations': ...} 2016-04-21 12:19:53,671 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/ssl-client.xml 2016-04-21 12:19:53,671 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/ssl-client.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,677 - Directory['/usr/hdp/2.3.4.0-3485/hadoop/conf/secure'] {'owner': 'root', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-04-21 12:19:53,677 - XmlConfig['ssl-client.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf/secure', 'configuration_attributes': {}, 'configurations': ...} 2016-04-21 12:19:53,686 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/secure/ssl-client.xml 2016-04-21 12:19:53,686 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/secure/ssl-client.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,692 - XmlConfig['ssl-server.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.4.0-3485/hadoop/conf', 'configuration_attributes': {}, 'configurations': ...} 2016-04-21 12:19:53,700 - Generating config: /usr/hdp/2.3.4.0-3485/hadoop/conf/ssl-server.xml 2016-04-21 12:19:53,700 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/ssl-server.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'} 2016-04-21 12:19:53,706 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/ssl-client.xml.example'] {'owner': 'mapred', 'group': 'hadoop'} 2016-04-21 12:19:53,707 - File['/usr/hdp/2.3.4.0-3485/hadoop/conf/ssl-server.xml.example'] {'owner': 'mapred', 'group': 'hadoop'} 2016-04-21 12:19:53,708 - File['/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid'] {'action': ['delete'], 'not_if': "ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'"} 2016-04-21 12:19:53,848 - Execute['ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/2.3.4.0-3485/hadoop/libexec && /usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh --config /usr/hdp/2.3.4.0-3485/hadoop/conf start nodemanager'] {'not_if': "ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'", 'user': 'yarn'} 2016-04-21 12:19:55,193 - Execute['ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`''] {'not_if': "ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'", 'tries': 5, 'try_sleep': 1} 2016-04-21 12:19:55,361 - Skipping Execute['ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`''] due to not_if 2016-04-21 12:19:55,362 - Executing NodeManager Stack Upgrade post-restart 2016-04-21 12:19:55,364 - NodeManager executing "yarn node -list -states=RUNNING" to verify the node has rejoined the cluster... 2016-04-21 12:19:55,364 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:19:58,034 - call returned (0, '16/04/21 12:19:56 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:19:56 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:19:58,036 - Will retry 29 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:20:08,046 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:20:10,746 - call returned (0, '16/04/21 12:20:09 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:20:09 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:20:10,746 - Will retry 28 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:20:20,757 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:20:24,086 - call returned (0, '16/04/21 12:20:22 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:20:22 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:20:24,087 - Will retry 27 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:20:34,097 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:20:36,498 - call returned (0, '16/04/21 12:20:35 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:20:35 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:20:36,498 - Will retry 26 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:20:46,504 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:20:49,100 - call returned (0, '16/04/21 12:20:47 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:20:48 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:20:49,101 - Will retry 25 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:20:59,104 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:21:01,843 - call returned (0, '16/04/21 12:21:00 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:21:00 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:21:01,844 - Will retry 24 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:21:11,854 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:21:14,480 - call returned (0, '16/04/21 12:21:13 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:21:13 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:21:14,481 - Will retry 23 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:21:24,489 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:21:27,073 - call returned (0, '16/04/21 12:21:25 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:21:25 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:21:27,074 - Will retry 22 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:21:37,083 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:21:39,681 - call returned (0, '16/04/21 12:21:38 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:21:38 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:21:39,682 - Will retry 21 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:21:49,687 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:21:52,359 - call returned (0, '16/04/21 12:21:51 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:21:51 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:21:52,359 - Will retry 20 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:22:02,367 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:22:04,996 - call returned (0, '16/04/21 12:22:03 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:22:03 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:22:04,997 - Will retry 19 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:22:15,007 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:22:17,600 - call returned (0, '16/04/21 12:22:16 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:22:16 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:22:17,600 - Will retry 18 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:22:27,605 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:22:30,172 - call returned (0, '16/04/21 12:22:28 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:22:29 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:22:30,172 - Will retry 17 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:22:40,176 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:22:42,772 - call returned (0, '16/04/21 12:22:41 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:22:41 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:22:42,773 - Will retry 16 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:22:52,783 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:22:55,391 - call returned (0, '16/04/21 12:22:54 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:22:54 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:22:55,405 - Will retry 15 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:23:05,408 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:23:08,135 - call returned (0, '16/04/21 12:23:06 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:23:06 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:23:08,136 - Will retry 14 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:23:18,147 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:23:20,866 - call returned (0, '16/04/21 12:23:19 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:23:19 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:23:20,867 - Will retry 13 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:23:30,874 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:23:33,507 - call returned (0, '16/04/21 12:23:32 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:23:32 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:23:33,508 - Will retry 12 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:23:43,518 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:23:46,181 - call returned (0, '16/04/21 12:23:44 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:23:45 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:23:46,182 - Will retry 11 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:23:56,192 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:23:58,840 - call returned (0, '16/04/21 12:23:57 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:23:57 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:23:58,841 - Will retry 10 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:24:08,846 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:24:11,757 - call returned (0, '16/04/21 12:24:10 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:24:10 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:24:11,758 - Will retry 9 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:24:21,767 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:24:24,336 - call returned (0, '16/04/21 12:24:23 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:24:23 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:24:24,337 - Will retry 8 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:24:34,347 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:24:36,999 - call returned (0, '16/04/21 12:24:35 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:24:35 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:24:37,000 - Will retry 7 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:24:47,010 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:24:49,460 - call returned (0, '16/04/21 12:24:48 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:24:48 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:24:49,478 - Will retry 6 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:24:59,489 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:25:02,106 - call returned (0, '16/04/21 12:25:00 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:25:00 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:25:02,107 - Will retry 5 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:25:12,117 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:25:15,078 - call returned (0, '16/04/21 12:25:13 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:25:13 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:25:15,079 - Will retry 4 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:25:25,090 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:25:27,728 - call returned (0, '16/04/21 12:25:26 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:25:26 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:25:27,728 - Will retry 3 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:25:37,735 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:25:40,249 - call returned (0, '16/04/21 12:25:39 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:25:39 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:25:40,250 - Will retry 2 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:25:50,260 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:25:52,893 - call returned (0, '16/04/21 12:25:51 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:25:51 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0') 2016-04-21 12:25:52,894 - Will retry 1 time(s), caught exception: NodeManager with ID node1.domain.net:45454 was not found in the list of running NodeManagers. Sleeping for 10 sec(s) 2016-04-21 12:26:02,904 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'} 2016-04-21 12:26:05,561 - call returned (0, '16/04/21 12:26:04 INFO impl.TimelineClientImpl: Timeline service address: http://node2.domain.net:8188/ws/v1/timeline/\n16/04/21 12:26:04 INFO client.RMProxy: Connecting to ResourceManager at node2.domain.net/13.111.111.11:8050\nTotal Nodes:2\n Node-Id\t Node-State\tNode-Http-Address\tNumber-of-Running-Containers\nnode1:45454\t RUNNING\tnode1:8042\t 0\nnode2:45454\t RUNNING\tnode2:8042\t 0')