Failed to start Services (HDFS , YARN , MAPPREEDUCE2 , HIVE , HBASE ...) after a successufull Instalation of HDP-3.1.0.0 and ambari 2.7.3.0 on a 4 hosts CLUSTER

[#1]

Hello, I just installed HDP-3.1.0.0 and amber 2.7.3.0, while creating my cluster with the ambari wizard, I chose the 3.1.0 version knowing that my VM's are running on Ubuntu 16.04.
The installation is successfully done, but warning shows up showing that services can't start!
here an example of during installation:

and a little bit after:

After complete with the wizard I tried to start manually those services (data, node in one of my hosts, for example, and it work for seconds before turning off).
Can anyone help?

[#2]

Previously, the above question was originally posted in the Community Help Track. On Tue May 28 23:07 UTC 2019, a member of the HCC moderation staff moved it to the Cloud & Operations track. The Community Help Track is intended for questions about using the HCC site itself.

[#3]

Hi @Adil BAKKOURI

After installation, when service were starting, one of your service start failed. Can you share logs?

Regards,

Vinay K

[#4]

Hi @Vinay,

wich log files you want me to share ? /var/log/ ?

after failled i got Alert on Ambari :

the hostname -f in evry host give the exact FQDN in the capture, is all my issues caused by this error 111 ?

Merci

[#5]

@Adil BAKKOURI

Just curious did you disable the FW or follow the steps in HDP preparing the environment

[#6]

Yes all HDP preparation are done:

[#7]

@Adil BAKKOURI

In the ambari UI can you drill down the Timeline Service V2.0 Reader start this will give an indication of what the problem is, please can you share the output

[#8]

that's the log when i try to start Timeline Service V2.0 Reader ==> LOG.txt

[#9]

hi @Geoffrey Shelton Okot,

that's the log when i try to start Timeline Service V2.0 Reader .

stderr: /var/lib/ambari-agent/data/errors-358.txt

Traceback (most recent call last):  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 119, in <module>    ApplicationTimelineReader().execute()  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute    method(env)  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 45, in start    self.configure(env) # FOR SECURITY  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 73, in configure    configure_hbase(env)  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 89, in configure_hbase    owner=params.yarn_hbase_user  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__    self.env.run()  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run    self.run_action(resource, action)  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action    provider_action()  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 677, in action_create_on_execute    self.action_delayed("create")  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 674, in action_delayed    self.get_hdfs_resource_executor().action_delayed(action_name, self)  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 373, in action_delayed    self.action_delayed_for_nameservice(None, action_name, main_resource)  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 395, in action_delayed_for_nameservice    self._assert_valid()  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 334, in _assert_valid    self.target_status = self._get_file_status(target)  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 497, in _get_file_status    list_status = self.util.run_command(target, 'GETFILESTATUS', method='GET', ignore_status_codes=['404'], assertable_result=False)  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 214, in run_command    return self._run_command(*args, **kwargs)  File "/usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py", line 282, in _run_command    _, out, err = get_user_call_output(cmd, user=self.run_user, logoutput=self.logoutput, quiet=False)  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/get_user_call_output.py", line 62, in get_user_call_output    raise ExecutionFailed(err_msg, code, files_output[0], files_output[1]) resource_management.core.exceptions.ExecutionFailed: Execution of 'curl -sS -L -w '%{http_code}' -X GET -d '' -H 'Content-Length: 0' 'http://master.rh.bigdata.cluster:50070/webhdfs/v1/atsv2/hbase/data?op=GETFILESTATUS&user.name=hdfs' 1>/tmp/tmpFiFkPa 2>/tmp/tmp9jUaKL' returned 7. curl: (7) Failed to connect to master.rh.bigdata.cluster port 50070: Connection refused 000

stdout: /var/lib/ambari-agent/data/output-358.txt

2019-05-29 11:50:12,000 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78 2019-05-29 11:50:12,040 - Using hadoop conf dir: /usr/hdp/3.1.0.0-78/hadoop/conf 2019-05-29 11:50:12,646 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78 2019-05-29 11:50:12,661 - Using hadoop conf dir: /usr/hdp/3.1.0.0-78/hadoop/conf 2019-05-29 11:50:12,665 - Group['hdfs'] {} 2019-05-29 11:50:12,670 - Group['hadoop'] {} 2019-05-29 11:50:12,670 - Group['users'] {} 2019-05-29 11:50:12,671 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2019-05-29 11:50:12,674 - User['yarn-ats'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2019-05-29 11:50:12,677 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2019-05-29 11:50:12,679 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2019-05-29 11:50:12,681 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None} 2019-05-29 11:50:12,682 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None} 2019-05-29 11:50:12,684 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop'], 'uid': None} 2019-05-29 11:50:12,686 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2019-05-29 11:50:12,688 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2019-05-29 11:50:12,690 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None} 2019-05-29 11:50:12,691 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2019-05-29 11:50:12,694 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'} 2019-05-29 11:50:12,752 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if 2019-05-29 11:50:12,760 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'} 2019-05-29 11:50:12,771 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2019-05-29 11:50:12,777 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2019-05-29 11:50:12,778 - call['/var/lib/ambari-agent/tmp/changeUid.sh hbase'] {} 2019-05-29 11:50:12,875 - call returned (0, '1031') 2019-05-29 11:50:12,888 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1031'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'} 2019-05-29 11:50:12,953 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1031'] due to not_if 2019-05-29 11:50:12,967 - Group['hdfs'] {} 2019-05-29 11:50:12,975 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop', u'hdfs']} 2019-05-29 11:50:12,981 - FS Type: HDFS 2019-05-29 11:50:12,982 - Directory['/etc/hadoop'] {'mode': 0755} 2019-05-29 11:50:13,047 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2019-05-29 11:50:13,050 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777} 2019-05-29 11:50:13,206 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'} 2019-05-29 11:50:13,310 - Skipping Execute[('setenforce', '0')] due to not_if 2019-05-29 11:50:13,318 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'} 2019-05-29 11:50:13,345 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'} 2019-05-29 11:50:13,350 - Directory['/var/run/hadoop/hdfs'] {'owner': 'hdfs', 'cd_access': 'a'} 2019-05-29 11:50:13,351 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'} 2019-05-29 11:50:13,375 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'} 2019-05-29 11:50:13,380 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'} 2019-05-29 11:50:13,397 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:13,424 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2019-05-29 11:50:13,426 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755} 2019-05-29 11:50:13,427 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'} 2019-05-29 11:50:13,437 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:13,509 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755} 2019-05-29 11:50:13,559 - Skipping unlimited key JCE policy check and setup since it is not required 2019-05-29 11:50:14,790 - Using hadoop conf dir: /usr/hdp/3.1.0.0-78/hadoop/conf 2019-05-29 11:50:14,792 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.0.0-78 -> 3.1.0.0-78 2019-05-29 11:50:14,836 - Using hadoop conf dir: /usr/hdp/3.1.0.0-78/hadoop/conf 2019-05-29 11:50:14,857 - Directory['/var/log/hadoop-yarn'] {'group': 'hadoop', 'cd_access': 'a', 'create_parents': True, 'ignore_failures': True, 'mode': 0775, 'owner': 'yarn'} 2019-05-29 11:50:14,862 - Directory['/var/run/hadoop-yarn'] {'owner': 'yarn', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:14,863 - Directory['/var/run/hadoop-yarn/yarn'] {'owner': 'yarn', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:14,864 - Directory['/var/log/hadoop-yarn/yarn'] {'owner': 'yarn', 'group': 'hadoop', 'create_parents': True, 'cd_access': 'a'} 2019-05-29 11:50:14,865 - Directory['/var/run/hadoop-mapreduce'] {'owner': 'mapred', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:14,866 - Directory['/var/run/hadoop-mapreduce/mapred'] {'owner': 'mapred', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:14,868 - Directory['/var/log/hadoop-mapreduce'] {'owner': 'mapred', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:14,870 - Directory['/var/log/hadoop-mapreduce/mapred'] {'owner': 'mapred', 'group': 'hadoop', 'create_parents': True, 'cd_access': 'a'} 2019-05-29 11:50:14,871 - Directory['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase'] {'owner': 'yarn-ats', 'group': 'hadoop', 'create_parents': True, 'cd_access': 'a'} 2019-05-29 11:50:14,873 - Directory['/var/run/hadoop-yarn-hbase'] {'owner': 'yarn-ats', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:14,873 - Creating directory Directory['/var/run/hadoop-yarn-hbase'] since it doesn't exist. 2019-05-29 11:50:14,874 - Changing owner for /var/run/hadoop-yarn-hbase from 0 to yarn-ats 2019-05-29 11:50:14,875 - Directory['/var/run/hadoop-yarn-hbase/yarn-ats'] {'owner': 'yarn-ats', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:14,875 - Creating directory Directory['/var/run/hadoop-yarn-hbase/yarn-ats'] since it doesn't exist. 2019-05-29 11:50:14,876 - Changing owner for /var/run/hadoop-yarn-hbase/yarn-ats from 0 to yarn-ats 2019-05-29 11:50:14,877 - Directory['/var/log/hadoop-yarn/embedded-yarn-ats-hbase'] {'owner': 'yarn-ats', 'group': 'hadoop', 'create_parents': True, 'cd_access': 'a'} 2019-05-29 11:50:14,877 - Creating directory Directory['/var/log/hadoop-yarn/embedded-yarn-ats-hbase'] since it doesn't exist. 2019-05-29 11:50:14,878 - Changing owner for /var/log/hadoop-yarn/embedded-yarn-ats-hbase from 0 to yarn-ats 2019-05-29 11:50:14,879 - Directory['/tmp'] {'create_parents': True, 'cd_access': 'a'} 2019-05-29 11:50:14,879 - Execute[('chmod', '1777', u'/tmp')] {'sudo': True} 2019-05-29 11:50:14,948 - XmlConfig['hbase-policy.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn-ats', 'configurations': {u'security.admin.protocol.acl': u'*', u'security.masterregion.protocol.acl': u'*', u'security.client.protocol.acl': u'*'}} 2019-05-29 11:50:15,008 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-policy.xml 2019-05-29 11:50:15,009 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-policy.xml'] {'owner': 'yarn-ats', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,017 - Writing File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-policy.xml'] because it doesn't exist 2019-05-29 11:50:15,018 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-policy.xml from 0 to yarn-ats 2019-05-29 11:50:15,030 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-env.sh'] {'content': InlineTemplate(...), 'owner': 'yarn-ats', 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:15,032 - Writing File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-env.sh'] because it doesn't exist 2019-05-29 11:50:15,032 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-env.sh from 0 to yarn-ats 2019-05-29 11:50:15,040 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase_grant_permissions.sh'] {'content': Template('yarn_hbase_grant_permissions.j2'), 'owner': 'yarn-ats', 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:15,041 - Writing File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase_grant_permissions.sh'] because it doesn't exist 2019-05-29 11:50:15,042 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase_grant_permissions.sh from 0 to yarn-ats 2019-05-29 11:50:15,057 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'yarn-ats', 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:15,058 - Writing File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/log4j.properties'] because it doesn't exist 2019-05-29 11:50:15,058 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/log4j.properties from 0 to yarn-ats 2019-05-29 11:50:15,069 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hadoop-metrics2-hbase.properties'] {'content': Template('hadoop-metrics2-hbase.properties.j2'), 'owner': 'yarn-ats', 'group': 'hadoop'} 2019-05-29 11:50:15,070 - Writing File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hadoop-metrics2-hbase.properties'] because it doesn't exist 2019-05-29 11:50:15,070 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hadoop-metrics2-hbase.properties from 0 to yarn-ats 2019-05-29 11:50:15,084 - Directory['/usr/lib/ambari-logsearch-logfeeder/conf'] {'create_parents': True, 'mode': 0755, 'cd_access': 'a'} 2019-05-29 11:50:15,085 - Generate Log Feeder config file: /usr/lib/ambari-logsearch-logfeeder/conf/input.config-yarn.json 2019-05-29 11:50:15,086 - File['/usr/lib/ambari-logsearch-logfeeder/conf/input.config-yarn.json'] {'content': Template('input.config-yarn.json.j2'), 'mode': 0644} 2019-05-29 11:50:15,088 - XmlConfig['core-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {u'final': {u'fs.defaultFS': u'true'}}, 'owner': 'hdfs', 'configurations': ...} 2019-05-29 11:50:15,113 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/core-site.xml 2019-05-29 11:50:15,114 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/core-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,179 - XmlConfig['hdfs-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {u'final': {u'dfs.datanode.failed.volumes.tolerated': u'true', u'dfs.datanode.data.dir': u'true', u'dfs.namenode.http-address': u'true', u'dfs.namenode.name.dir': u'true', u'dfs.webhdfs.enabled': u'true'}}, 'owner': 'hdfs', 'configurations': ...} 2019-05-29 11:50:15,198 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/hdfs-site.xml 2019-05-29 11:50:15,198 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/hdfs-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,273 - XmlConfig['mapred-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...} 2019-05-29 11:50:15,287 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/mapred-site.xml 2019-05-29 11:50:15,288 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/mapred-site.xml'] {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,365 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/mapred-site.xml from 1030 to yarn 2019-05-29 11:50:15,366 - XmlConfig['yarn-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {u'hidden': {u'hadoop.registry.dns.bind-port': u'true'}}, 'owner': 'yarn', 'configurations': ...} 2019-05-29 11:50:15,382 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/yarn-site.xml 2019-05-29 11:50:15,383 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/yarn-site.xml'] {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,655 - XmlConfig['capacity-scheduler.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...} 2019-05-29 11:50:15,673 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/capacity-scheduler.xml 2019-05-29 11:50:15,674 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/capacity-scheduler.xml'] {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,703 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/capacity-scheduler.xml from 1028 to yarn 2019-05-29 11:50:15,704 - XmlConfig['hbase-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn-ats', 'configurations': ...} 2019-05-29 11:50:15,724 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-site.xml 2019-05-29 11:50:15,725 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/embedded-yarn-ats-hbase/hbase-site.xml'] {'owner': 'yarn-ats', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,794 - XmlConfig['resource-types.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': {u'yarn.resource-types.yarn.io_gpu.maximum-allocation': u'8', u'yarn.resource-types': u''}} 2019-05-29 11:50:15,810 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/resource-types.xml 2019-05-29 11:50:15,811 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/resource-types.xml'] {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:15,818 - File['/etc/security/limits.d/yarn.conf'] {'content': Template('yarn.conf.j2'), 'mode': 0644} 2019-05-29 11:50:15,823 - File['/etc/security/limits.d/mapreduce.conf'] {'content': Template('mapreduce.conf.j2'), 'mode': 0644} 2019-05-29 11:50:15,848 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/yarn-env.sh'] {'content': InlineTemplate(...), 'owner': 'yarn', 'group': 'hadoop', 'mode': 0755} 2019-05-29 11:50:15,854 - File['/usr/hdp/3.1.0.0-78/hadoop-yarn/bin/container-executor'] {'group': 'hadoop', 'mode': 02050} 2019-05-29 11:50:15,871 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/container-executor.cfg'] {'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:15,872 - Directory['/cgroups_test/cpu'] {'group': 'hadoop', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'} 2019-05-29 11:50:15,880 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/mapred-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'mode': 0755} 2019-05-29 11:50:15,883 - Directory['/var/log/hadoop-yarn/nodemanager/recovery-state'] {'owner': 'yarn', 'group': 'hadoop', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'} 2019-05-29 11:50:15,888 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/taskcontroller.cfg'] {'content': Template('taskcontroller.cfg.j2'), 'owner': 'hdfs'} 2019-05-29 11:50:15,890 - XmlConfig['mapred-site.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'mapred', 'configurations': ...} 2019-05-29 11:50:15,914 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/mapred-site.xml 2019-05-29 11:50:15,915 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/mapred-site.xml'] {'owner': 'mapred', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:16,043 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/mapred-site.xml from 1029 to mapred 2019-05-29 11:50:16,044 - XmlConfig['capacity-scheduler.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hdfs', 'configurations': ...} 2019-05-29 11:50:16,068 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/capacity-scheduler.xml 2019-05-29 11:50:16,069 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/capacity-scheduler.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:16,092 - Changing owner for /usr/hdp/3.1.0.0-78/hadoop/conf/capacity-scheduler.xml from 1029 to hdfs 2019-05-29 11:50:16,093 - XmlConfig['ssl-client.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hdfs', 'configurations': ...} 2019-05-29 11:50:16,109 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/ssl-client.xml 2019-05-29 11:50:16,109 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/ssl-client.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:16,119 - Directory['/usr/hdp/3.1.0.0-78/hadoop/conf/secure'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'cd_access': 'a'} 2019-05-29 11:50:16,121 - XmlConfig['ssl-client.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf/secure', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hdfs', 'configurations': ...} 2019-05-29 11:50:16,137 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/secure/ssl-client.xml 2019-05-29 11:50:16,138 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/secure/ssl-client.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:16,147 - XmlConfig['ssl-server.xml'] {'group': 'hadoop', 'conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'hdfs', 'configurations': ...} 2019-05-29 11:50:16,162 - Generating config: /usr/hdp/3.1.0.0-78/hadoop/conf/ssl-server.xml 2019-05-29 11:50:16,163 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/ssl-server.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'} 2019-05-29 11:50:16,175 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/ssl-client.xml.example'] {'owner': 'mapred', 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:16,177 - File['/usr/hdp/3.1.0.0-78/hadoop/conf/ssl-server.xml.example'] {'owner': 'mapred', 'group': 'hadoop', 'mode': 0644} 2019-05-29 11:50:16,185 - HdfsResource['/atsv2/hbase/data'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.0.0-78/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'hdfs://master.rh.bigdata.cluster:8020', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'user': 'hdfs', 'owner': 'yarn-ats', 'hadoop_conf_dir': '/usr/hdp/3.1.0.0-78/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp']} 2019-05-29 11:50:16,205 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://master.rh.bigdata.cluster:50070/webhdfs/v1/atsv2/hbase/data?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpFiFkPa 2>/tmp/tmp9jUaKL''] {'logoutput': None, 'quiet': False} 2019-05-29 11:50:16,417 - call returned (7, '') 
Command failed after 1 tries

[#10]

@Adil BAKKOURI the above response was submitted twice. The second submission was rejected by a member of the HCC moderation staff because it was a duplicate.

[#11]

Thank You @Bill Brooks

[#12]

Hi @Vinay , @Geoffrey Shelton Okot,
Any updates or solutions for this problem ?
Thank You