I faced similar issue, but REST api thru -   http://localhost:8080/api/v1/cluster/[cluster_name]/services/KERBEROS  was not available, and kerberos remove was giving error - "The 'krb5-conf' configuration is not available" in ambari-server.log   We had Ambari 2.7.5.0. In that version clusterconfigmapping is not present, we had to do below procedure:   In ambari db -  psql -U ambari -d ambari (default password: bigdata) ambari=> select config_id, selected, create_timestamp from clusterconfig where type_name in ('kerberos-env', 'krb5-conf'); config_id | selected | create_timestamp -----------+----------+------------------ 352 | 0 | 1589796215822 452 | 0 | 1589800820822 << Select the record with latest timestamp 353 | 0 | 1589796215857 453 | 0 | 1589800820858 <<Select the record with latest timestamp (4 rows) ambari=> update clusterconfig set selected=1 where config_id=452; UPDATE 1 ambari=> update clusterconfig set selected=1 where config_id=453; UPDATE 1 ambari=> update clusterconfig set selected=1 where config_id=452; update clusterconfig set selected=1 where config_id=453;   * Still the kerberos REST api was not been shown. * Remove the kerberos * Add kerberos back ... View more

@Elvis Zhang Also it will be good to see if you can start the DataNode without any issue manually using the following command: # su hdfs -l -s /bin/bash -c 'ulimit -c unlimited ; /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config /usr/hdp/current/hadoop-client/conf start datanode' . ... View more

@Elvis Zhang You can avoid Ambari UI to ask password for kadmin. You can store that kadmin credential to ambari store. https://community.hortonworks.com/articles/42927/adding-kdc-administrator-credentials-to-the-ambari.html curl -H "X-Requested-By:ambari" -u admin:admin -X PUT -d '{ "Credential" : { "principal" : "admin/admin@EXAMPLE.COM", "key" : "pwd$hwx", "type" : "persisted" } }' http://ambari.example.com:8080/api/v1/clusters/c1/credentials/kdc.admin.credential . Above will require a keystore setup and then "[2] Encrypt passwords stored in ambari.properties file." on ambari-server setup-security wizard. Using this way you can avoid entering the kadmin credentials everytime on ambari. ... View more

yes i checked "/etc/krb5.conf", again ,nothing wrong ,from error text ,it means "*.keytabs" file used old "EXAMPLE.COM", I don't know how to update or rebuild the keytabs? ... View more

stderr: /var/lib/ambari-agent/data/errors-2515.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 81, in <module> KerberosServiceCheck().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 64, in service_check user=params.smoke_user File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/kinit -c /var/lib/ambari-agent/tmp/kerberos_service_check_cc_2ce6caa1397f604948063293c5e53d22 -kt /etc/security/keytabs/kerberos.service_check.032217.keytab hadoop-032217@EXAMPLE.COM' returned 1. kinit: Cannot find KDC for realm "EXAMPLE.COM" while getting initial credentials stdout: /var/lib/ambari-agent/data/output-2515.txt Performing kinit using hadoop-032217@EXAMPLE.COM 2017-03-22 08:47:04,347 - Execute['/usr/bin/kinit -c /var/lib/ambari-agent/tmp/kerberos_service_check_cc_2ce6caa1397f604948063293c5e53d22 -kt /etc/security/keytabs/kerberos.service_check.032217.keytab hadoop-032217@EXAMPLE.COM'] {'user': 'ambari-qa'} 2017-03-22 08:47:04,417 - File['/var/lib/ambari-agent/tmp/kerberos_service_check_cc_2ce6caa1397f604948063293c5e53d22'] {'action': ['delete']} Command failed after 1 tries ... View more

OK i fix it . My OS is 'Centos 7.1', i alos used (Centos 6.*) service iptables stop So i change is with systemctl stop firewalld Every thing is ok Now , Thanks for Jay's help again. ... View more