Hi. Im deploying hdp 2..6.1 in rhel 7 i have installed ambari. and when deploying the cluster, i get the following error , at the first step of the deploy ---- Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 35, in <module> BeforeAnyHook().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 29, in hook setup_users() File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py", line 51, in setup_users fetch_nonlocal_groups = params.fetch_nonlocal_groups, File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/accounts.py", line 84, in action_create shell.checked_call(command, sudo=True) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'usermod -G hdfs -g hadoop hdfs' returned 6. usermod: user 'hdfs' does not exist in /etc/passwd Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-3626.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-3626.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py", line 37, in <module> BeforeInstallHook().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 382, in execute self.save_component_version_to_structured_out(self.command_name) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 244, in save_component_version_to_structured_out stack_select_package_name = stack_select.get_package_name() File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/stack_select.py", line 110, in get_package_name package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/stack_select.py", line 224, in get_packages supported_packages = get_supported_packages() File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/stack_select.py", line 148, in get_supported_packages raise Fail("Unable to query for supported packages using {0}".format(stack_selector_path)) resource_management.core.exceptions.Fail: Unable to query for supported packages using /usr/bin/hdp-select stdout: /var/lib/ambari-agent/data/output-3626.txt 2018-05-14 14:54:16,835 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=None -> 2.6 2018-05-14 14:54:16,841 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf 2018-05-14 14:54:16,843 - Group['livy'] {} 2018-05-14 14:54:16,844 - Group['spark'] {} 2018-05-14 14:54:16,844 - Group['hdfs'] {} 2018-05-14 14:54:16,844 - Group['hadoop'] {} 2018-05-14 14:54:16,844 - Group['users'] {} 2018-05-14 14:54:16,845 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None} 2018-05-14 14:54:16,853 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None} 2018-05-14 14:54:16,860 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None} 2018-05-14 14:54:16,866 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None} 2018-05-14 14:54:16,872 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None} 2018-05-14 14:54:16,879 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None} 2018-05-14 14:54:16,885 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None} 2018-05-14 14:54:16,891 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None} 2018-05-14 14:54:16,898 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs'], 'uid': None} 2018-05-14 14:54:16,904 - Modifying user hdfs Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-3626.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-3626.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', ''] Command failed after 1 --------------------------------------------- all the users exist but not the hdfs user. eny ideas? thanks ... View more

HI, ive upgrades HDP from 2.4 to 2.5 to 2.6. now kafka is at 0.10.1 . im have Kerberos configured and working with Ranger I can create , produce , and consuming with zookeeper ( usinf kinit and a user that have permissions at Ranger rule) WORKING OK ./kafka-console-producer.sh --broker-list ctl-t01.my.domain:6667,ctl-t02.my.domain:6667,ctl-t01.my.domain:6667 --topic bptopic --security-protocol PLAINTEXTSASL ./kafka-console-consumer.sh --bootstrap-server ctl-t01.my.domain:2181:6667 --topic bptopic --security-protocol PLAINTEXTSASL --from-beginning when i use the Brokers instead of zookeper , the consumer , is not consuming NOT WORKING ./kafka-console-consumer.sh --zookeeper ctl-t01.my.domain:2181 --topic bptopic--security-protocol PLAINTEXTSASL --from-beginning i have cheked logs , and documentation , without any solution. Could be , that i need to delete at zookepper rmr /brokers? ( im not sure, but in another environment , that was the solution. deleting all the topics..). i want to see if i can keep all the topics , since we keep them for more than a week, and deleting them will be the last resource. thanks much @Ian Roberts ... View more