1.not all the service can't start 2.I have setup the "sudore" permission properly for every ambari-agent as well as mentioned in "ambari_agent_for_non-root.html" 3.zookeeper can start 4. Start DN error logs bellow stderr: /var/lib/ambari-agent/data/errors-3731.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py", line 174, in <module> DataNode().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 720, in restart self.start(env, upgrade_type=upgrade_type) File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py", line 61, in start datanode(action="start") File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk return fn(*args, **kwargs) File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py", line 68, in datanode create_log_dir=True File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py", line 269, in service Execute(daemon_cmd, not_if=process_id_exists_command, environment=hadoop_env_exports) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh su hdfs -l -s /bin/bash -c 'ulimit -c unlimited ; /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config /usr/hdp/current/hadoop-client/conf start datanode'' returned 1. starting datanode, logging to /var/log/hadoop/hdfs/hadoop-hdfs-datanode-hadoop-namenode-1.out stdout: /var/lib/ambari-agent/data/output-3731.txt 2017-03-28 15:59:11,350 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.4.0.0-169 2017-03-28 15:59:11,352 - Checking if need to create versioned ... View more

yes ,i change user to ambari , the error gone. but another quesion occured hadoop's datanode and NameNode can't be start . resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh su hdfs -l -s /bin/bash -c 'ulimit -c unlimited ; /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config ... View more

Thanks @Jay SenSharma @Robert Levas ，i find the reason when click Regenerate Keytab, a web page pop, Principle and password needed . but some time it pop up,some time nothing。i will find why . ... View more

yes, it happened every time when click 'Regenreate Keytab'. the Ambari version is 2.4.2.0-136 ... View more

1. ambari is running as ambari i think the reason is "STDERR: kadmin: Cannot read password while initializing kadmin interface" but when i run kadmin at another host（not kdc server) , i can connect the kdc server like this（input kadmin password） . they are all same with kdc's. maybe the ambari-server read password from error config ?? ... View more

thanks Jay very much! i works ;what's the reason of this error ... View more

so many rows in tables, last version in 'clusterconfigmapping ' table mysql>select * from clusterconfig WHERE type_name in ('kerberos-env', 'krb5-conf'); | 1059 | version1490153129291 | 41 | krb5-conf | 2 | 5 | {"domains":"","manage_krb5_conf":"false","conf_dir":"/etc","content":" ***** | 1490153128147 | ******* 82 rows in set (0.02 sec) mysql> select * from clusterconfigmapping WHERE type_name in ('kerberos-env', 'krb5-conf'); +--------------+------------------+------------+----------+----------------------+-----------+ | type_name | create_timestamp | cluster_id | selected | version_tag | user_name | +--------------+------------------+------------+----------+----------------------+-----------+ | krb5-conf | 1490153128175 | 2 | 0 | version1490153129291 | admin | +--------------+------------------+------------+----------+----------------------+-----------+ mysql> select ccm.type_name, ccm.version_tag, ccm.selected, cc.version_tag from clusterconfigmapping ccm left join clusterconfig cc on ccm.version_tag = cc.version_tag where ccm.selected = 1 and cc.version_tag is NULL; Empty set (0.00 sec) should i change table 'clusterconfigmapping' ,the column 'selected=1' like this update clusterconfigmapping set select=1 where version_tag='version1490153129291'and type_name='krb5-conf' ... View more

stderr: /var/lib/ambari-agent/data/errors-2515.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 81, in <module> KerberosServiceCheck().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 64, in service_check user=params.smoke_user File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/kinit -c /var/lib/ambari-agent/tmp/kerberos_service_check_cc_2ce6caa1397f604948063293c5e53d22 -kt /etc/security/keytabs/kerberos.service_check.032217.keytab hadoop-032217@EXAMPLE.COM' returned 1. kinit: Cannot find KDC for realm "EXAMPLE.COM" while getting initial credentials stdout: /var/lib/ambari-agent/data/output-2515.txt Performing kinit using hadoop-032217@EXAMPLE.COM 2017-03-22 08:47:04,347 - Execute['/usr/bin/kinit -c /var/lib/ambari-agent/tmp/kerberos_service_check_cc_2ce6caa1397f604948063293c5e53d22 -kt /etc/security/keytabs/kerberos.service_check.032217.keytab hadoop-032217@EXAMPLE.COM'] {'user': 'ambari-qa'} 2017-03-22 08:47:04,417 - File['/var/lib/ambari-agent/tmp/kerberos_service_check_cc_2ce6caa1397f604948063293c5e53d22'] {'action': ['delete']} Command failed after 1 tries ... View more

i try refesh chrome ,clean all cookies . but i didn't work. i check ambari-server.log ,nothing related with kerberos ... View more

yes i checked "/etc/krb5.conf", again ,nothing wrong ,from error text ,it means "*.keytabs" file used old "EXAMPLE.COM", I don't know how to update or rebuild the keytabs? ... View more

when i delete file /etc/security/keytabs/kerberos.service_check.032117.keytab another error occured like this: stderr: /var/lib/ambari-agent/data/errors-2362.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 81, in <module> KerberosServiceCheck().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 72, in service_check raise Fail(err_msg) resource_management.core.exceptions.Fail: Failed to execute kinit test due to principal or keytab not found or available stdout: /var/lib/ambari-agent/data/output-2362.txt Command failed after 1 tries ... View more

Kerberos didn't sucess, there no "Regenerate Keytabs". i don't know how to regenrate hte keytabs? ... View more

OK i fix it . My OS is 'Centos 7.1', i alos used (Centos 6.*) service iptables stop So i change is with systemctl stop firewalld Every thing is ok Now , Thanks for Jay's help again. ... View more

thanks Jay very much !! yes ,i find an error in ambari-metrics-monitor.out [WARNING] emitter.py:84 - Error sending metrics to server. [Errno 113] No route to host it seems that metrics-collector host can't be connect ,but i don't know how to config this ? does it ok when i installed by ambari-server? here is my hosts in '/etc/hosts' (all hosts are same) 192.168.2.43 bn-001.hadoop 192.168.2.39 bn-002.hadoop 192.168.2.38 bn-003.hadoop ( this is ambari-metrics-collector) here is “ metric_monitor.ini ” in my "metric collector host is "bn-001.hadoop" [default] debug_level = INFO hostname = bn-001.hadoop enable_time_threshold = false enable_value_threshold = false [emitter] send_interval = 60 [collector] collector_sleep_interval = 10 max_queue_size = 5000 host = bn-003.hadoop port = 6188 https_enabled = False ... View more

yes, i checked them , they were all installed on three hosts. ... View more