Failed to start HiveServer2 using Ambari 2.7 and HDP 3.1

[#1]

Changes that i tried but did not help:

1. set home directory for java
2. changed the warehouse root directory
3. created the znode for hiveserver2 manually (but it returns empty set)
4. reinstalled zookeeper and hive again

stderr: /var/lib/ambari-agent/data/errors-252.txt

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_server.py", line 143, in <module>
    HiveServer().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_server.py", line 53, in start
    hive_service('hiveserver2', action = 'start', upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_service.py", line 101, in hive_service
    wait_for_znode()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/decorator.py", line 54, in wrapper
    return function(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_service.py", line 184, in wait_for_znode
    raise Exception(format("HiveServer2 is no longer running, check the logs a

stdout: /var/lib/ambari-agent/data/output-252.txt

2019-04-25 17:15:42,610 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:42,635 - Using hadoop conf dir: /usr/hdp/3.1.0.0-78/hadoop/conf
2019-04-25 17:15:42,871 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:42,878 - Using hadoop conf dir: /usr/hdp/3.1.0.0-78/hadoop/conf
2019-04-25 17:15:42,880 - Group['livy'] {}
2019-04-25 17:15:42,881 - Group['spark'] {}
2019-04-25 17:15:42,881 - Group['ranger'] {}
2019-04-25 17:15:42,882 - Group['hdfs'] {}
2019-04-25 17:15:42,882 - Group['zeppelin'] {}
2019-04-25 17:15:42,882 - Group['hadoop'] {}
2019-04-25 17:15:42,882 - Group['users'] {}
2019-04-25 17:15:42,883 - User['yarn-ats'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,884 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,885 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,886 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,888 - User['superset'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,889 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2019-04-25 17:15:42,890 - User['atlas'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,891 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,892 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['ranger', 'hadoop'], 'uid': None}
2019-04-25 17:15:42,893 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2019-04-25 17:15:42,894 - User['zeppelin'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['zeppelin', 'hadoop'], 'uid': None}
2019-04-25 17:15:42,895 - User['accumulo'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,896 - User['logsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,897 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['livy', 'hadoop'], 'uid': None}
2019-04-25 17:15:42,898 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['spark', 'hadoop'], 'uid': None}
2019-04-25 17:15:42,900 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2019-04-25 17:15:42,901 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,902 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop'], 'uid': None}
2019-04-25 17:15:42,903 - User['sqoop'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,904 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,905 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,906 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-04-25 17:15:42,907 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2019-04-25 17:15:42,909 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2019-04-25 17:15:42,917 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2019-04-25 17:15:42,917 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}
2019-04-25 17:15:42,919 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2019-04-25 17:15:42,920 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2019-04-25 17:15:42,921 - call['/var/lib/ambari-agent/tmp/changeUid.sh hbase'] {}
2019-04-25 17:15:42,933 - call returned (0, '1022')
2019-04-25 17:15:42,935 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1022'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2019-04-25 17:15:42,941 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1022'] due to not_if
2019-04-25 17:15:42,942 - Group['hdfs'] {}
2019-04-25 17:15:42,943 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop', u'hdfs']}
2019-04-25 17:15:42,943 - FS Type: HDFS
2019-04-25 17:15:42,943 - Directory['/etc/hadoop'] {'mode': 0755}
2019-04-25 17:15:42,964 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2019-04-25 17:15:42,965 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2019-04-25 17:15:42,987 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2019-04-25 17:15:43,096 - Skipping Execute[('setenforce', '0')] due to only_if
2019-04-25 17:15:43,097 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'}
2019-04-25 17:15:43,103 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'}
2019-04-25 17:15:43,105 - Directory['/var/run/hadoop/hdfs'] {'owner': 'hdfs', 'cd_access': 'a'}
2019-04-25 17:15:43,106 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'}
2019-04-25 17:15:43,116 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2019-04-25 17:15:43,150 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2019-04-25 17:15:43,187 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:43,220 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2019-04-25 17:15:43,237 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2019-04-25 17:15:43,328 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2019-04-25 17:15:43,338 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:43,364 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2019-04-25 17:15:43,379 - Skipping unlimited key JCE policy check and setup since it is not required
2019-04-25 17:15:43,993 - Using hadoop conf dir: /usr/hdp/3.1.0.0-78/hadoop/conf
2019-04-25 17:15:44,005 - call['ambari-python-wrap /usr/bin/hdp-select status hive-server2'] {'timeout': 20}
2019-04-25 17:15:44,035 - call returned (0, 'hive-server2 - 3.1.0.0-78')
2019-04-25 17:15:44,037 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:44,065 - File['/var/lib/ambari-agent/cred/lib/CredentialUtil.jar'] {'content': DownloadSource('http://infra.toodevops.com:8080/resources/CredentialUtil.jar'), 'mode': 0755}
2019-04-25 17:15:44,067 - Not downloading the file from http://infra.toodevops.com:8080/resources/CredentialUtil.jar, because /var/lib/ambari-agent/tmp/CredentialUtil.jar already exists
2019-04-25 17:15:45,381 - Directories to fill with configs: [u'/usr/hdp/current/hive-server2/conf', u'/usr/hdp/current/hive-server2/conf/']
2019-04-25 17:15:45,381 - Directory['/etc/hive/3.1.0.0-78/0'] {'owner': 'hive', 'group': 'hadoop', 'create_parents': True, 'mode': 0755}
2019-04-25 17:15:45,383 - XmlConfig['mapred-site.xml'] {'group': 'hadoop', 'conf_dir': '/etc/hive/3.1.0.0-78/0', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hive', 'configurations': ...}
2019-04-25 17:15:45,399 - Generating config: /etc/hive/3.1.0.0-78/0/mapred-site.xml
2019-04-25 17:15:45,399 - File['/etc/hive/3.1.0.0-78/0/mapred-site.xml'] {'owner': 'hive', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2019-04-25 17:15:45,457 - File['/etc/hive/3.1.0.0-78/0/hive-default.xml.template'] {'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,458 - File['/etc/hive/3.1.0.0-78/0/hive-env.sh.template'] {'owner': 'hive', 'group': 'hadoop', 'mode': 0755}
2019-04-25 17:15:45,462 - File['/etc/hive/3.1.0.0-78/0/llap-daemon-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,466 - File['/etc/hive/3.1.0.0-78/0/llap-cli-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,469 - File['/etc/hive/3.1.0.0-78/0/hive-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,472 - File['/etc/hive/3.1.0.0-78/0/hive-exec-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,475 - File['/etc/hive/3.1.0.0-78/0/beeline-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,476 - XmlConfig['beeline-site.xml'] {'owner': 'hive', 'group': 'hadoop', 'mode': 0644, 'conf_dir': '/etc/hive/3.1.0.0-78/0', 'configurations': {'beeline.hs2.jdbc.url.container': u'jdbc:hive2://infra.toodevops.com:2181,infraha.toodevops.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2', 'beeline.hs2.jdbc.url.default': u'container'}}
2019-04-25 17:15:45,487 - Generating config: /etc/hive/3.1.0.0-78/0/beeline-site.xml
2019-04-25 17:15:45,488 - File['/etc/hive/3.1.0.0-78/0/beeline-site.xml'] {'owner': 'hive', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2019-04-25 17:15:45,491 - File['/etc/hive/3.1.0.0-78/0/parquet-logging.properties'] {'content': ..., 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,492 - Directory['/etc/hive/3.1.0.0-78/0'] {'owner': 'hive', 'group': 'hadoop', 'create_parents': True, 'mode': 0755}
2019-04-25 17:15:45,492 - XmlConfig['mapred-site.xml'] {'group': 'hadoop', 'conf_dir': '/etc/hive/3.1.0.0-78/0', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hive', 'configurations': ...}
2019-04-25 17:15:45,503 - Generating config: /etc/hive/3.1.0.0-78/0/mapred-site.xml
2019-04-25 17:15:45,504 - File['/etc/hive/3.1.0.0-78/0/mapred-site.xml'] {'owner': 'hive', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2019-04-25 17:15:45,561 - File['/etc/hive/3.1.0.0-78/0/hive-default.xml.template'] {'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,562 - File['/etc/hive/3.1.0.0-78/0/hive-env.sh.template'] {'owner': 'hive', 'group': 'hadoop', 'mode': 0755}
2019-04-25 17:15:45,567 - File['/etc/hive/3.1.0.0-78/0/llap-daemon-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,570 - File['/etc/hive/3.1.0.0-78/0/llap-cli-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,573 - File['/etc/hive/3.1.0.0-78/0/hive-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,576 - File['/etc/hive/3.1.0.0-78/0/hive-exec-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,578 - File['/etc/hive/3.1.0.0-78/0/beeline-log4j2.properties'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,579 - XmlConfig['beeline-site.xml'] {'owner': 'hive', 'group': 'hadoop', 'mode': 0644, 'conf_dir': '/etc/hive/3.1.0.0-78/0', 'configurations': {'beeline.hs2.jdbc.url.container': u'jdbc:hive2://infra.toodevops.com:2181,infraha.toodevops.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2', 'beeline.hs2.jdbc.url.default': u'container'}}
2019-04-25 17:15:45,591 - Generating config: /etc/hive/3.1.0.0-78/0/beeline-site.xml
2019-04-25 17:15:45,591 - File['/etc/hive/3.1.0.0-78/0/beeline-site.xml'] {'owner': 'hive', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2019-04-25 17:15:45,594 - File['/etc/hive/3.1.0.0-78/0/parquet-logging.properties'] {'content': ..., 'owner': 'hive', 'group': 'hadoop', 'mode': 0644}
2019-04-25 17:15:45,595 - File['/usr/hdp/current/hive-server2/conf/hive-site.jceks'] {'content': StaticFile('/var/lib/ambari-agent/cred/conf/hive_server/hive-site.jceks'), 'owner': 'hive', 'group': 'hadoop', 'mode': 0640}
2019-04-25 17:15:45,596 - Writing File['/usr/hdp/current/hive-server2/conf/hive-site.jceks'] because contents don't match
2019-04-25 17:15:45,597 - XmlConfig['hive-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/current/hive-server2/conf/', 'mode': 0644, 'configuration_attributes': {u'hidden': {u'javax.jdo.option.ConnectionPassword': u'HIVE_CLIENT,CONFIG_DOWNLOAD'}}, 'owner': 'hive', 'configurations': ...}
2019-04-25 17:15:45,607 - Generating config: /usr/hdp/current/hive-server2/conf/hive-site.xml
2019-04-25 17:15:45,607 - File['/usr/hdp/current/hive-server2/conf/hive-site.xml'] {'owner': 'hive', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2019-04-25 17:15:45,784 - Writing File['/usr/hdp/current/hive-server2/conf/hive-site.xml'] because contents don't match
2019-04-25 17:15:45,785 - Generating Atlas Hook config file /usr/hdp/current/hive-server2/conf/atlas-application.properties
2019-04-25 17:15:45,785 - PropertiesFile['/usr/hdp/current/hive-server2/conf/atlas-application.properties'] {'owner': 'hive', 'group': 'hadoop', 'mode': 0644, 'properties': ...}
2019-04-25 17:15:45,790 - Generating properties file: /usr/hdp/current/hive-server2/conf/atlas-application.properties
2019-04-25 17:15:45,790 - File['/usr/hdp/current/hive-server2/conf/atlas-application.properties'] {'owner': 'hive', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2019-04-25 17:15:45,807 - Writing File['/usr/hdp/current/hive-server2/conf/atlas-application.properties'] because contents don't match
2019-04-25 17:15:45,813 - File['/usr/hdp/current/hive-server2/conf//hive-env.sh'] {'content': InlineTemplate(...), 'owner': 'hive', 'group': 'hadoop', 'mode': 0755}
2019-04-25 17:15:45,813 - Writing File['/usr/hdp/current/hive-server2/conf//hive-env.sh'] because contents don't match
2019-04-25 17:15:45,814 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2019-04-25 17:15:45,817 - File['/etc/security/limits.d/hive.conf'] {'content': Template('hive.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2019-04-25 17:15:45,818 - File['/usr/lib/ambari-agent/DBConnectionVerification.jar'] {'content': DownloadSource('http://infra.toodevops.com:8080/resources/DBConnectionVerification.jar'), 'mode': 0644}
2019-04-25 17:15:45,819 - Not downloading the file from http://infra.toodevops.com:8080/resources/DBConnectionVerification.jar, because /var/lib/ambari-agent/tmp/DBConnectionVerification.jar already exists
2019-04-25 17:15:45,819 - Directory['/var/run/hive'] {'owner': 'hive', 'create_parents': True, 'group': 'hadoop', 'mode': 0755, 'cd_access': 'a'}
2019-04-25 17:15:45,820 - Directory['/var/log/hive'] {'owner': 'hive', 'create_parents': True, 'group': 'hadoop', 'mode': 0755, 'cd_access': 'a'}
2019-04-25 17:15:45,820 - Directory['/var/lib/hive'] {'owner': 'hive', 'create_parents': True, 'group': 'hadoop', 'mode': 0755, 'cd_access': 'a'}
2019-04-25 17:15:45,830 - File['/var/lib/ambari-agent/tmp/start_hiveserver2_script'] {'content': Template('startHiveserver2.sh.j2'), 'mode': 0755}
2019-04-25 17:15:45,859 - File['/usr/hdp/current/hive-server2/conf/hadoop-metrics2-hiveserver2.properties'] {'content': Template('hadoop-metrics2-hiveserver2.properties.j2'), 'owner': 'hive', 'group': 'hadoop', 'mode': 0600}
2019-04-25 17:15:45,870 - XmlConfig['hiveserver2-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/current/hive-server2/conf/', 'mode': 0600, 'configuration_attributes': {}, 'owner': 'hive', 'configurations': ...}
2019-04-25 17:15:45,881 - Generating config: /usr/hdp/current/hive-server2/conf/hiveserver2-site.xml
2019-04-25 17:15:45,881 - File['/usr/hdp/current/hive-server2/conf/hiveserver2-site.xml'] {'owner': 'hive', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0600, 'encoding': 'UTF-8'}
2019-04-25 17:15:45,893 - Called copy_to_hdfs tarball: mapreduce
2019-04-25 17:15:45,893 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:45,893 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:15:45,894 - Source file: /usr/hdp/3.1.0.0-78/hadoop/mapreduce.tar.gz , Dest file in HDFS: /hdp/apps/3.1.0.0-78/mapreduce/mapreduce.tar.gz
2019-04-25 17:15:45,894 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:45,894 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:15:45,894 - HdfsResource['/hdp/apps/3.1.0.0-78/mapreduce'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0555}
2019-04-25 17:15:45,898 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/mapreduce?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpIHx4bN 2>/tmp/tmpxqJSGm''] {'logoutput': None, 'quiet': False}
2019-04-25 17:15:46,067 - call returned (0, '')
2019-04-25 17:15:46,068 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":2,"fileId":16415,"group":"hdfs","length":0,"modificationTime":1556118410835,"owner":"hdfs","pathSuffix":"","permission":"555","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:15:46,070 - HdfsResource['/hdp/apps/3.1.0.0-78/mapreduce/mapreduce.tar.gz'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'source': '/usr/hdp/3.1.0.0-78/hadoop/mapreduce.tar.gz', 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'replace_existing_files': False, 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'group': 'hadoop', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'file', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0444}
2019-04-25 17:15:46,072 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/mapreduce/mapreduce.tar.gz?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmp3q5vl3 2>/tmp/tmpfbGZeF''] {'logoutput': None, 'quiet': False}
2019-04-25 17:15:46,235 - call returned (0, '')
2019-04-25 17:15:46,235 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":1556117415821,"blockSize":134217728,"childrenNum":0,"fileId":16426,"group":"hadoop","length":308401145,"modificationTime":1556117419759,"owner":"hdfs","pathSuffix":"","permission":"444","replication":3,"storagePolicy":0,"type":"FILE"}}200', u'')
2019-04-25 17:15:46,237 - DFS file /hdp/apps/3.1.0.0-78/mapreduce/mapreduce.tar.gz is identical to /usr/hdp/3.1.0.0-78/hadoop/mapreduce.tar.gz, skipping the copying
2019-04-25 17:15:46,237 - Will attempt to copy mapreduce tarball from /usr/hdp/3.1.0.0-78/hadoop/mapreduce.tar.gz to DFS at /hdp/apps/3.1.0.0-78/mapreduce/mapreduce.tar.gz.
2019-04-25 17:15:46,237 - Called copy_to_hdfs tarball: tez
2019-04-25 17:15:46,237 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:46,238 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:15:46,238 - Source file: /usr/hdp/3.1.0.0-78/tez/lib/tez.tar.gz , Dest file in HDFS: /hdp/apps/3.1.0.0-78/tez/tez.tar.gz
2019-04-25 17:15:46,238 - Preparing the Tez tarball...
2019-04-25 17:15:46,238 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:46,238 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:15:46,239 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:15:46,239 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:15:46,239 - Extracting /usr/hdp/3.1.0.0-78/hadoop/mapreduce.tar.gz to /var/lib/ambari-agent/tmp/mapreduce-tarball-AKNIox
2019-04-25 17:15:46,240 - Execute[('tar', '-xf', u'/usr/hdp/3.1.0.0-78/hadoop/mapreduce.tar.gz', '-C', '/var/lib/ambari-agent/tmp/mapreduce-tarball-AKNIox/')] {'tries': 3, 'sudo': True, 'try_sleep': 1}
2019-04-25 17:15:53,050 - Extracting /usr/hdp/3.1.0.0-78/tez/lib/tez.tar.gz to /var/lib/ambari-agent/tmp/tez-tarball-E9LkDJ
2019-04-25 17:15:53,051 - Execute[('tar', '-xf', u'/usr/hdp/3.1.0.0-78/tez/lib/tez.tar.gz', '-C', '/var/lib/ambari-agent/tmp/tez-tarball-E9LkDJ/')] {'tries': 3, 'sudo': True, 'try_sleep': 1}
2019-04-25 17:15:58,727 - Execute[('cp', '-a', '/var/lib/ambari-agent/tmp/mapreduce-tarball-AKNIox/hadoop/lib/native', '/var/lib/ambari-agent/tmp/tez-tarball-E9LkDJ/lib')] {'sudo': True}
2019-04-25 17:15:58,899 - Directory['/var/lib/ambari-agent/tmp/tez-tarball-E9LkDJ/lib'] {'recursive_ownership': True, 'mode': 0755, 'cd_access': 'a'}
2019-04-25 17:15:58,900 - Creating a new Tez tarball at /var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz
2019-04-25 17:15:58,901 - Execute[('tar', '-zchf', '/tmp/tmpno811A', '-C', '/var/lib/ambari-agent/tmp/tez-tarball-E9LkDJ', '.')] {'tries': 3, 'sudo': True, 'try_sleep': 1}
2019-04-25 17:16:16,988 - Execute[('mv', '/tmp/tmpno811A', '/var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz')] {}
2019-04-25 17:16:17,868 - HdfsResource['/hdp/apps/3.1.0.0-78/tez'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0555}
2019-04-25 17:16:17,871 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/tez?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmp6FxIsD 2>/tmp/tmpjwQErS''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:19,259 - call returned (0, '')
2019-04-25 17:16:19,259 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16433,"group":"hdfs","length":0,"modificationTime":1556117444130,"owner":"hdfs","pathSuffix":"","permission":"555","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:19,262 - HdfsResource['/hdp/apps/3.1.0.0-78/tez/tez.tar.gz'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'source': '/var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz', 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'replace_existing_files': False, 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'group': 'hadoop', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'file', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0444}
2019-04-25 17:16:19,265 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/tez/tez.tar.gz?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpJ_MZ1g 2>/tmp/tmpR2Ys4u''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:19,408 - call returned (0, '')
2019-04-25 17:16:19,409 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":1556117444130,"blockSize":134217728,"childrenNum":0,"fileId":16434,"group":"hadoop","length":254858532,"modificationTime":1556117446004,"owner":"hdfs","pathSuffix":"","permission":"444","replication":3,"storagePolicy":0,"type":"FILE"}}200', u'')
2019-04-25 17:16:19,410 - Not replacing existing DFS file /hdp/apps/3.1.0.0-78/tez/tez.tar.gz which is different from /var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz, due to replace_existing_files=False
2019-04-25 17:16:19,411 - Will attempt to copy tez tarball from /var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz to DFS at /hdp/apps/3.1.0.0-78/tez/tez.tar.gz.
2019-04-25 17:16:19,411 - Called copy_to_hdfs tarball: pig
2019-04-25 17:16:19,412 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:16:19,412 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:16:19,412 - pig-env is not present on the cluster. Skip copying /usr/hdp/3.1.0.0-78/pig/pig.tar.gz
2019-04-25 17:16:19,412 - Called copy_to_hdfs tarball: hive
2019-04-25 17:16:19,413 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:16:19,413 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:16:19,413 - Source file: /usr/hdp/3.1.0.0-78/hive/hive.tar.gz , Dest file in HDFS: /hdp/apps/3.1.0.0-78/hive/hive.tar.gz
2019-04-25 17:16:19,417 - HdfsResource['/hdp/apps/3.1.0.0-78/hive'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0555}
2019-04-25 17:16:19,420 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/hive?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpeangH2 2>/tmp/tmpIK6nfs''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:19,557 - call returned (0, '')
2019-04-25 17:16:19,558 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":17694,"group":"hdfs","length":0,"modificationTime":1556118404431,"owner":"hdfs","pathSuffix":"","permission":"555","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:19,561 - HdfsResource['/hdp/apps/3.1.0.0-78/hive/hive.tar.gz'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'source': '/usr/hdp/3.1.0.0-78/hive/hive.tar.gz', 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'replace_existing_files': False, 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'group': 'hadoop', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'file', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0444}
2019-04-25 17:16:19,564 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/hive/hive.tar.gz?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmptbu0YX 2>/tmp/tmpS8fxqG''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:19,704 - call returned (0, '')
2019-04-25 17:16:19,705 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":1556118404431,"blockSize":134217728,"childrenNum":0,"fileId":17695,"group":"hadoop","length":363312102,"modificationTime":1556118407635,"owner":"hdfs","pathSuffix":"","permission":"444","replication":3,"storagePolicy":0,"type":"FILE"}}200', u'')
2019-04-25 17:16:19,706 - DFS file /hdp/apps/3.1.0.0-78/hive/hive.tar.gz is identical to /usr/hdp/3.1.0.0-78/hive/hive.tar.gz, skipping the copying
2019-04-25 17:16:19,707 - Will attempt to copy hive tarball from /usr/hdp/3.1.0.0-78/hive/hive.tar.gz to DFS at /hdp/apps/3.1.0.0-78/hive/hive.tar.gz.
2019-04-25 17:16:19,707 - Called copy_to_hdfs tarball: sqoop
2019-04-25 17:16:19,707 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:16:19,708 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:16:19,708 - Source file: /usr/hdp/3.1.0.0-78/sqoop/sqoop.tar.gz , Dest file in HDFS: /hdp/apps/3.1.0.0-78/sqoop/sqoop.tar.gz
2019-04-25 17:16:19,723 - HdfsResource['/hdp/apps/3.1.0.0-78/sqoop'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0555}
2019-04-25 17:16:19,726 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/sqoop?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpSuryao 2>/tmp/tmp3SD3xf''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:19,855 - call returned (0, '')
2019-04-25 17:16:19,856 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":17696,"group":"hdfs","length":0,"modificationTime":1556118409393,"owner":"hdfs","pathSuffix":"","permission":"555","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:19,858 - HdfsResource['/hdp/apps/3.1.0.0-78/sqoop/sqoop.tar.gz'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'source': '/usr/hdp/3.1.0.0-78/sqoop/sqoop.tar.gz', 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'replace_existing_files': False, 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'group': 'hadoop', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'file', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0444}
2019-04-25 17:16:19,860 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/sqoop/sqoop.tar.gz?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpOE0Pht 2>/tmp/tmp6ezsyy''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:19,990 - call returned (0, '')
2019-04-25 17:16:19,991 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":1556118409393,"blockSize":134217728,"childrenNum":0,"fileId":17697,"group":"hadoop","length":77261259,"modificationTime":1556118410012,"owner":"hdfs","pathSuffix":"","permission":"444","replication":3,"storagePolicy":0,"type":"FILE"}}200', u'')
2019-04-25 17:16:19,992 - DFS file /hdp/apps/3.1.0.0-78/sqoop/sqoop.tar.gz is identical to /usr/hdp/3.1.0.0-78/sqoop/sqoop.tar.gz, skipping the copying
2019-04-25 17:16:19,993 - Will attempt to copy sqoop tarball from /usr/hdp/3.1.0.0-78/sqoop/sqoop.tar.gz to DFS at /hdp/apps/3.1.0.0-78/sqoop/sqoop.tar.gz.
2019-04-25 17:16:19,993 - Called copy_to_hdfs tarball: hadoop_streaming
2019-04-25 17:16:19,993 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78
2019-04-25 17:16:19,994 - Tarball version was calcuated as 3.1.0.0-78. Use Command Version: True
2019-04-25 17:16:19,994 - Source file: /usr/hdp/3.1.0.0-78/hadoop-mapreduce/hadoop-streaming.jar , Dest file in HDFS: /hdp/apps/3.1.0.0-78/mapreduce/hadoop-streaming.jar
2019-04-25 17:16:20,014 - HdfsResource['/hdp/apps/3.1.0.0-78/mapreduce'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0555}
2019-04-25 17:16:20,016 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/mapreduce?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmp0A7Wch 2>/tmp/tmpr6wkVO''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:20,155 - call returned (0, '')
2019-04-25 17:16:20,156 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":2,"fileId":16415,"group":"hdfs","length":0,"modificationTime":1556118410835,"owner":"hdfs","pathSuffix":"","permission":"555","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:20,158 - HdfsResource['/hdp/apps/3.1.0.0-78/mapreduce/hadoop-streaming.jar'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'source': '/usr/hdp/3.1.0.0-78/hadoop-mapreduce/hadoop-streaming.jar', 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'replace_existing_files': False, 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hdfs', 'group': 'hadoop', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'file', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0444}
2019-04-25 17:16:20,160 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/hdp/apps/3.1.0.0-78/mapreduce/hadoop-streaming.jar?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpOwd8tq 2>/tmp/tmpl5tpx7''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:20,325 - call returned (0, '')
2019-04-25 17:16:20,325 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":1556118410835,"blockSize":134217728,"childrenNum":0,"fileId":17699,"group":"hadoop","length":176342,"modificationTime":1556118410880,"owner":"hdfs","pathSuffix":"","permission":"444","replication":3,"storagePolicy":0,"type":"FILE"}}200', u'')
2019-04-25 17:16:20,327 - DFS file /hdp/apps/3.1.0.0-78/mapreduce/hadoop-streaming.jar is identical to /usr/hdp/3.1.0.0-78/hadoop-mapreduce/hadoop-streaming.jar, skipping the copying
2019-04-25 17:16:20,327 - Will attempt to copy hadoop_streaming tarball from /usr/hdp/3.1.0.0-78/hadoop-mapreduce/hadoop-streaming.jar to DFS at /hdp/apps/3.1.0.0-78/mapreduce/hadoop-streaming.jar.
2019-04-25 17:16:20,328 - HdfsResource['/warehouse/tablespace/external/hive/sys.db/'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hive', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 01755}
2019-04-25 17:16:20,331 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/warehouse/tablespace/external/hive/sys.db/?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpHv8SZA 2>/tmp/tmp1e9n25''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:20,478 - call returned (0, '')
2019-04-25 17:16:20,479 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"aclBit":true,"blockSize":0,"childrenNum":46,"fileId":17411,"group":"hadoop","length":0,"modificationTime":1556118413452,"owner":"hive","pathSuffix":"","permission":"1755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:20,481 - HdfsResource['/warehouse/tablespace/external/hive/sys.db/query_data/'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hive', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 01777}
2019-04-25 17:16:20,483 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/warehouse/tablespace/external/hive/sys.db/query_data/?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmplK9QuQ 2>/tmp/tmph7VNEG''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:20,610 - call returned (0, '')
2019-04-25 17:16:20,610 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"aclBit":true,"blockSize":0,"childrenNum":1,"fileId":17412,"group":"hadoop","length":0,"modificationTime":1556117682872,"owner":"hive","pathSuffix":"","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:20,612 - HdfsResource['/warehouse/tablespace/external/hive/sys.db/dag_meta'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hive', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 01777}
2019-04-25 17:16:20,615 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/warehouse/tablespace/external/hive/sys.db/dag_meta?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpDU6pQE 2>/tmp/tmpsQfo4d''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:20,741 - call returned (0, '')
2019-04-25 17:16:20,742 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"aclBit":true,"blockSize":0,"childrenNum":0,"fileId":17700,"group":"hadoop","length":0,"modificationTime":1556118412147,"owner":"hive","pathSuffix":"","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:20,744 - HdfsResource['/warehouse/tablespace/external/hive/sys.db/dag_data'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hive', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 01777}
2019-04-25 17:16:20,747 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/warehouse/tablespace/external/hive/sys.db/dag_data?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpewir7_ 2>/tmp/tmp1tZC2W''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:20,873 - call returned (0, '')
2019-04-25 17:16:20,874 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"aclBit":true,"blockSize":0,"childrenNum":0,"fileId":17701,"group":"hadoop","length":0,"modificationTime":1556118412808,"owner":"hive","pathSuffix":"","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:20,876 - HdfsResource['/warehouse/tablespace/external/hive/sys.db/app_data'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'owner': 'hive', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 01777}
2019-04-25 17:16:20,879 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://infra.toodevops.com:50070/webhdfs/v1/warehouse/tablespace/external/hive/sys.db/app_data?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmplicaU4 2>/tmp/tmpkNy9C7''] {'logoutput': None, 'quiet': False}
2019-04-25 17:16:21,008 - call returned (0, '')
2019-04-25 17:16:21,009 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"aclBit":true,"blockSize":0,"childrenNum":2,"fileId":17702,"group":"hadoop","length":0,"modificationTime":1556162039260,"owner":"hive","pathSuffix":"","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2019-04-25 17:16:21,011 - HdfsResource[None] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://infra.toodevops.com:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': 'missing_principal', 'user': 'hdfs', 'action': ['execute'], 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp']}
2019-04-25 17:16:21,093 - Directory['/usr/lib/ambari-logsearch-logfeeder/conf'] {'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2019-04-25 17:16:21,094 - Generate Log Feeder config file: /usr/lib/ambari-logsearch-logfeeder/conf/input.config-hive.json
2019-04-25 17:16:21,095 - File['/usr/lib/ambari-logsearch-logfeeder/conf/input.config-hive.json'] {'content': Template('input.config-hive.json.j2'), 'mode': 0644}
2019-04-25 17:16:21,096 - Ranger Hive plugin is not enabled
2019-04-25 17:16:21,098 - call['ambari-sudo.sh su hive -l -s /bin/bash -c 'cat /var/run/hive/hive-server.pid 1>/tmp/tmp6VURUS 2>/tmp/tmpdOqSp5''] {'quiet': False}
2019-04-25 17:16:21,241 - call returned (0, '')
2019-04-25 17:16:21,241 - get_user_call_output returned (0, u'56793', u'')
2019-04-25 17:16:21,243 - call['ambari-sudo.sh su hive -l -s /bin/bash -c 'hive --config /usr/hdp/current/hive-server2/conf/ --service metatool -listFSRoot' 2>/dev/null | grep hdfs:// | cut -f1,2,3 -d '/' | grep -v 'hdfs://infra.toodevops.com:8020' | head -1'] {}
2019-04-25 17:16:50,419 - call returned (0, '')
2019-04-25 17:16:50,419 - Execute['/var/lib/ambari-agent/tmp/start_hiveserver2_script /var/log/hive/hive-server2.out /var/log/hive/hive-server2.err /var/run/hive/hive-server.pid /usr/hdp/current/hive-server2/conf/ /etc/tez/conf'] {'environment': {'HIVE_BIN': 'hive', 'JAVA_HOME': u'/usr/jdk64/jdk1.8.0_112', 'HADOOP_HOME': u'/usr/hdp/current/hadoop-client'}, 'not_if': 'ls /var/run/hive/hive-server.pid >/dev/null 2>&1 && ps -p 56793 >/dev/null 2>&1', 'user': 'hive', 'path': [u'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hive-server2/bin:/usr/hdp/3.1.0.0-78/hadoop/bin']}
2019-04-25 17:16:50,695 - Execute['/usr/jdk64/jdk1.8.0_112/bin/java -cp /usr/lib/ambari-agent/DBConnectionVerification.jar:/usr/hdp/current/hive-server2/lib/mysql-connector-java-5.1.47.jar org.apache.ambari.server.DBConnectionVerification 'jdbc:mysql://compute.toodevops.com/hive?createDatabaseIfNotExist=true' bigmehr [PROTECTED] com.mysql.jdbc.Driver'] {'path': ['/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 5, 'try_sleep': 10}
2019-04-25 17:16:52,141 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:16:53,583 - call returned (1, '')
2019-04-25 17:16:53,584 - Will retry 29 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:17:03,603 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:17:04,719 - call returned (1, '')
2019-04-25 17:17:04,737 - Will retry 28 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:17:14,747 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:17:16,884 - call returned (1, '')
2019-04-25 17:17:16,934 - Will retry 27 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:17:26,945 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:17:28,665 - call returned (1, '')
2019-04-25 17:17:28,666 - Will retry 26 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:17:38,678 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:17:39,800 - call returned (1, '')
2019-04-25 17:17:39,802 - Will retry 25 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:17:49,812 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:17:51,546 - call returned (1, '')
2019-04-25 17:17:51,581 - Will retry 24 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:18:01,591 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:18:03,177 - call returned (1, '')
2019-04-25 17:18:03,178 - Will retry 23 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:18:13,194 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:18:14,074 - call returned (1, '')
2019-04-25 17:18:14,075 - Will retry 22 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:18:24,086 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:18:25,394 - call returned (1, '')
2019-04-25 17:18:25,396 - Will retry 21 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:18:35,407 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:18:36,669 - call returned (1, '')
2019-04-25 17:18:36,670 - Will retry 20 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:18:46,680 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:18:47,730 - call returned (1, '')
2019-04-25 17:18:47,732 - Will retry 19 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:18:57,744 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:18:58,132 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:18:58,144 - Will retry 18 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:19:08,154 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:19:10,209 - call returned (1, '')
2019-04-25 17:19:10,210 - Will retry 17 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:19:20,294 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:19:21,714 - call returned (1, '')
2019-04-25 17:19:21,731 - Will retry 16 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:19:31,742 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:19:33,071 - call returned (1, '')
2019-04-25 17:19:33,072 - Will retry 15 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:19:43,083 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:19:44,414 - call returned (1, '')
2019-04-25 17:19:44,415 - Will retry 14 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:19:54,433 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:19:54,750 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:19:54,752 - Will retry 13 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:20:04,762 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:20:06,670 - call returned (1, '')
2019-04-25 17:20:06,673 - Will retry 12 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:20:16,684 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:20:18,175 - call returned (1, '')
2019-04-25 17:20:18,176 - Will retry 11 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:20:28,187 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:20:28,546 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:20:28,547 - Will retry 10 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:20:38,556 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:20:38,758 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:20:38,759 - Will retry 9 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:20:48,770 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:20:49,114 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:20:49,115 - Will retry 8 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:20:59,127 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:20:59,416 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:20:59,436 - Will retry 7 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:21:09,445 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:21:09,948 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:21:09,959 - Will retry 6 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:21:19,970 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server infraha.toodevops.com:2181,infra.toodevops.com:2181 ls /hiveserver2 | grep 'serverUri=''] {}
2019-04-25 17:21:20,352 - call returned (1, "Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000707000000, 130023424, 0) failed; error='Cannot allocate memory' (errno=12)")
2019-04-25 17:21:20,365 - Will retry 5 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 10 sec(s)
2019-04-25 17:21:30,378 - Process with pid 66871 is not running. Stale pid file at /var/run/hive/hive-server.pid

Command failed after 1 tries

cat /var/log/hive/hiveserver2.log | grep ERROR

2019-04-25T17:21:22,938 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Exception during rename2019-04-25T17:21:24,411 ERROR [main]: metrics2.CodahaleMetrics (:()) - Unable to instantiate using constructor(MetricRegistry, HiveConf) for reporter org.apache.hadoop.hive.common.metrics.metrics2.Metrics2Reporter from conf HIVE_CODAHALE_METRICS_REPORTER_CLASSES2019-04-25T17:21:24,438 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Unable to rename temp file /tmp/hmetrics221400335606640996json to /tmp/report.json2019-04-25T17:21:24,439 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Exception during rename2019-04-25T17:21:25,454 ERROR [main]: server.HiveServer2 (HiveServer2.java:start(740)) - Error starting Web UI:2019-04-25T17:21:25,611 ERROR [main]: server.HiveServer2 (HiveServer2.java:execute(1343)) - Error starting HiveServer22019-04-25T17:21:26,105 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Unable to rename temp file /tmp/hmetrics5682491866228312214json to /tmp/report.json2019-04-25T17:21:26,105 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Exception during rename2019-04-25T17:21:26,904 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Unable to rename temp file /tmp/hmetrics6263799357756058898json to /tmp/report.json2019-04-25T17:21:26,904 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Exception during rename

cat /var/log/hive/hiveserver2.log | grep WARN

2019-04-25T17:18:21,322 WARN  [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(1100)) - Error starting HiveServer2 on attempt 2, will retry in 60000ms
2019-04-25T17:18:21,385 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:18:21,390 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:18:21,480 WARN  [PrivilegeSynchronizer]: metastore.RetryingMetaStoreClient (:()) - MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. refresh_privileges
2019-04-25T17:19:21,670 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:19:21,671 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:19:21,687 WARN  [main]: impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - hiveserver2 metrics system already initialized!
2019-04-25T17:19:21,688 WARN  [main]: server.HiveServer2 (HiveServer2.java:init(209)) - Could not initiate the HiveServer2 Metrics system.  Metrics may not be reported.
2019-04-25T17:19:21,798 WARN  [main]: session.SessionState (:()) - METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
2019-04-25T17:19:21,941 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:19:21,942 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:19:22,309 WARN  [main-EventThread]: server.HiveServer2 (HiveServer2.java:process(652)) - This HiveServer2 instance is now de-registered from ZooKeeper. The server will be shut down after the last client session completes.
2019-04-25T17:19:22,310 WARN  [main-EventThread]: server.HiveServer2 (HiveServer2.java:process(660)) - This instance of HiveServer2 has been removed from the list of server instances available for dynamic service discovery. The last client session has ended - will shutdown now.
2019-04-25T17:19:22,386 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:19:22,386 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:19:22,398 WARN  [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(1100)) - Error starting HiveServer2 on attempt 3, will retry in 60000ms
2019-04-25T17:19:22,461 WARN  [HiveMaterializedViewsRegistry-0]: session.SessionState (:()) - METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
2019-04-25T17:19:22,621 WARN  [PrivilegeSynchronizer]: metastore.RetryingMetaStoreClient (:()) - MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. refresh_privileges
2019-04-25T17:20:22,835 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:20:22,836 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:20:22,845 WARN  [main]: impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - hiveserver2 metrics system already initialized!
2019-04-25T17:20:22,846 WARN  [main]: server.HiveServer2 (HiveServer2.java:init(209)) - Could not initiate the HiveServer2 Metrics system.  Metrics may not be reported.
2019-04-25T17:20:22,974 WARN  [main]: session.SessionState (:()) - METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
2019-04-25T17:20:23,172 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:20:23,172 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:20:23,611 WARN  [main-EventThread]: server.HiveServer2 (HiveServer2.java:process(652)) - This HiveServer2 instance is now de-registered from ZooKeeper. The server will be shut down after the last client session completes.
2019-04-25T17:20:23,611 WARN  [main-EventThread]: server.HiveServer2 (HiveServer2.java:process(660)) - This instance of HiveServer2 has been removed from the list of server instances available for dynamic service discovery. The last client session has ended - will shutdown now.
2019-04-25T17:20:23,632 WARN  [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(1100)) - Error starting HiveServer2 on attempt 4, will retry in 60000ms
2019-04-25T17:20:23,641 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:20:23,642 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:20:23,773 WARN  [PrivilegeSynchronizer]: metastore.RetryingMetaStoreClient (:()) - MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. refresh_privileges
2019-04-25T17:20:24,065 WARN  [HiveMaterializedViewsRegistry-0]: session.SessionState (:()) - METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
2019-04-25T17:21:24,392 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:21:24,393 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:21:24,410 WARN  [main]: impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - hiveserver2 metrics system already initialized!
2019-04-25T17:21:24,411 WARN  [main]: server.HiveServer2 (HiveServer2.java:init(209)) - Could not initiate the HiveServer2 Metrics system.  Metrics may not be reported.
2019-04-25T17:21:24,642 WARN  [main]: session.SessionState (:()) - METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
2019-04-25T17:21:24,980 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:21:24,981 WARN  [main]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:21:25,563 WARN  [main-EventThread]: server.HiveServer2 (HiveServer2.java:process(652)) - This HiveServer2 instance is now de-registered from ZooKeeper. The server will be shut down after the last client session completes.
2019-04-25T17:21:25,563 WARN  [main-EventThread]: server.HiveServer2 (HiveServer2.java:process(660)) - This instance of HiveServer2 has been removed from the list of server instances available for dynamic service discovery. The last client session has ended - will shutdown now.
2019-04-25T17:21:25,604 WARN  [HiveMaterializedViewsRegistry-0]: session.SessionState (:()) - METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
2019-04-25T17:21:25,718 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.stats.fetch.partition.stats does not exist
2019-04-25T17:21:25,718 WARN  [PrivilegeSynchronizer]: conf.HiveConf (HiveConf.java:initialize(5310)) - HiveConf of name hive.heapsize does not exist
2019-04-25T17:21:25,866 WARN  [PrivilegeSynchronizer]: metastore.RetryingMetaStoreClient (:()) - MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. refresh_privileges

[#2]

@Yegane Ahmadnejad

1. Don't manually set the java_home if you are on RHEL/Centos

tar xzf jdk-8u171-linux-x64.tar.gz
cd /opt/jdk1.8.0_171/
alternatives --install /usr/bin/java java /opt/jdk1.8.0_171/bin/java 2
alternatives --config java
alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_171/bin/jar 2
alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_171/bin/javac 2
alternatives --set jar /opt/jdk1.8.0_171/bin/jar
alternatives --set javac /opt/jdk1.8.0_171/bin/javac

2. Don't change the warehouse root directory

3. Don't create the hiveserver2 znode manually.

4. Didn't see the hive database setup step??

[#3 · Accepted solution]

My problem was although hive memory was configured correctly but can not allocated . (because node doesnt have sufficent memmory) so by increasing node memmory ,It is solved