Support Questions

Find answers, ask questions, and share your expertise

Failed to create principal - hadoop@domain - can not check if principal exists

avatar
Contributor

I am trying to ENABLE Kerberos. I entered KDC configs and then Kadmin_host, admin_principal and admin_password.

Then it is successfully installing Kerberos cliend but failing while testing Kerberos client with below error:

Failed to create principal - hadoop@domain - can not check if principal exists

-> I find this error in "ambari-audit.log"

I have a container in my Active Directory called KnoxUsers and I have "knxadmin" as user.

I have put 'knxadmin' as Admin_Principal while configuring Kerberos.

Can anyone points me where I need to check to correct this error?

Brief Description about my cluster:

I have single node cluster which I deployed using ambari on a VM on a host-machine. I have active directory setup on a domain controller. My domain controller and host-machine are on the same domain.

21 REPLIES 21

avatar
Contributor
@tsharma

I created kadm5.acl file with below lines - changed Example.COM to my domain:

/admin@ATHENA.MIT.EDU        *                       
joeadmin@ATHENA.MIT.EDU   ADMCIL                     
joeadmin/*@ATHENA.MIT.EDU il  */root@ATHENA.MIT.EDU  
*/root@ATHENA.MIT.EDU     cil *1@ATHENA.MIT.EDU      
*/*@ATHENA.MIT.EDU        i                          
*/admin@EXAMPLE.COM       x   * -maxlife 9h -postdateable

I restarted krb5-admin-server service.

In kadmin config, I changed Admin_principal to knxadmin/admin@MYDOMAIN but that didn't work. Still getting same error.

avatar
Super Collaborator

If you admin is knxadmin/admin@MYDOMAIN, just having

*/admin@MYDOMAIN *

is enough. Remove other lines and restart.

Also see whether kadmin.local -q 'listprincs' lists your admin principal.

avatar
Contributor

@tsharma

Still same error as I tried removing all the lines accept the last one.

Also, kadmin.local command is giving me error :

Authenticating as principat root/admin@mydomain with password.

kadmin.local: no such file or directory while initializing kadmin.local interface

avatar
Super Collaborator

Did you do this?

service krb5-kdc restart

service krb5-admin-server restart

Please check the status of above two services. (service krb5-kdc status). Restart them if they are not running.

If it's not installed, install again. apt-get install krb5-kdc krb5-admin-server

avatar
Super Collaborator

Also did you create kerberos database? If not, create it.

krb5_newrealm

Do check your /etc/krb5.conf again.

avatar
Master Mentor

@tsharma

You have a couple of things wrong in your Kerberos setup. I noticed in your cluster the REALM is MIT.EDU, it should be different and NOT as you stated MIT.EDU (Michigan Institue of Technology) creators of Kerberos.Your kadm5.acl is wrong ! Your REALM is not ATHENA.MIT.EDU, that an example given with the MIT documentation.

How did the container "KnoxUsers" and "knxadmin" user get created in AD?

Below is the procedure on a Centos/RHEL but the commands are similar on all UNIX/LINUX OS'es

Assumptions:

REALM is TEST.COM

Install the KDC server.

The below command will deliver the nessary configuration files

# yum install krb5-server 

Edit your /etc/krb5.conf replace all occurrences of TEST and test please match case(lower or upper)

# cat /etc/krb5.conf 

The krb5.conf should look like this please notice the entries in lowercase for test.com and .test.com

[libdefaults] 
renew_lifetime = 7d 
forwardable = true 
default_realm = TEST.COM 
ticket_lifetime = 24h 
dns_lookup_realm = false 
dns_lookup_kdc = false 
default_ccache_name = /tmp/krb5cc_%{uid} 
#default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 
#default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 

[domain_realm] 
test.com = TEST.COM 
.test.com = TEST.COM 

[logging] 
default = FILE:/var/log/krb5kdc.log 
admin_server = FILE:/var/log/kadmind.log 
kdc = FILE:/var/log/krb5kdc.log 

[realms] TEST.COM = 
{ admin_server = {your_kdc_server} 
kdc = {your_kdc_server} } 

Modify the kdc.conf in /var/kerberos/krb5kdc/kdc.conf replace the TEST.COM with your REALM

cat /var/kerberos/krb5kdc/kdc.conf 

The kdc.conf should look like this

[kdcdefaults] 
kdc_ports = 88 
kdc_tcp_ports = 88 

[realms] TEST.COM = 
{ #master_key_type = aes256-cts 
acl_file = /var/kerberos/krb5kdc/kadm5.acl 
dict_file = /usr/share/dict/words 
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab 
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal

Delete the existing KDC database (optional)

# kdb5_util destroy -f TEST.COM (delete Kerberos existing TEST database) 

The below will prompt you for the password please note them and don't lose it.

Create the KDC database is didn't already

# kdb5_util create -s TEST.COM (create Kerberos database) 

When the above process terminates you will have created a database for the TEST.COM (realm)

Start the kadmin & KDC

# service krb5kdc start 
# service kadmin start 

To autostart the KDC and Kadmin on system bootup execute the below step

# chkconfig krb5kdc on 
# chkconfig kadmin on 

The value for the kadm5.acl should match the output of the below command.

On the KDC as root run

# kdestroy (ONLY if you already created a previous REAL database) 

Create principal

# kadmin.local -q "addprinc admin/admin" 
Authenticating as principal root/admin@TEST with password. 
WARNING: no policy specified for admin/admin@TEST; 
defaulting to no policy Enter password for principal "admin/admin@TEST": 
Re-enter password for principal "admin/admin@TEST": 
Principal "admin/admin@TEST" created. 

Validate you can connect to the KDC with admin rights

# kadmin.local 
Authenticating as principal root/admin@TEST.COM with password. 

From the above your kadm5.acl in /var/kerberos/krb5kdc/kadm5.acl should be,notice the space between the.COM and *

*/admin@TEST.COM * 

Validate that get a valid kerberos ticket

# kinit admin/admin@TEST 
Password for admin/admin@TEST 

Check validity it should be 7 days according to your krb5.conf

# klist 
Ticket cache: FILE:/tmp/krb5cc_0 Default 
principal: admin/admin@TEST 
Valid starting          Expires                Service principal
10/13/2017 15:48:43     10/14/2017 15:48:43  krbtgt/TEST.COM@TEST.COM

Now you can trigger the Ambari -Kerberos wizard and walk through the steps, the input values you need are Admin principal and Admin principal password

The keytabs should now be generated successfully, in your case single node check on the VM in /etc/security/keytabs

Please let me know if that helped

avatar
Contributor

@Geoffrey Thank you so much, for detailed steps!

But before moving further, I have few questions (you might find dumb but would be great if you can help since I am a complete newbie):

1) My assumption was that the my KDC server is Domain Controller(windows server 2012) where my Active Directory Domain Services are running. In this case Why do I need to install KDC server on my ubuntu host machine where my hadoop is running?

2) Why do I need to create admin on my host machine? I thought the user I created on my Active Directory on DC i.e. knxadmin can be used to login as admin.

3) I created KnoxUser container and knxadmin user in that container by following below guideline:

https://www.ibm.com/support/knowledgecenter/SSPT3X_4.2.0/com.ibm.swg.im.infosphere.biginsights.admin...

I hope, its a right way to do that?

avatar
Master Mentor

@tsharma

Can you have a look and this HCC doc and get back to me. I think that's what you intend to implement.

Create MIT KDC in the Hadoop cluster to manage service principals while using a one-way trust to allow AD users to utilze the Hadoop environment seamlessly

avatar
Contributor

@Geoffrey

Actually, my intent is to use existing Active Directory. I think, document you pointed is when I intent to use Existing MIT KDC, which is not the case.

My existing Active Directory is setup already at Domain Controller (let's say, hostname=DC1) . And there I created new container and a admin user in that container. I want to use KDC as DC1 to connect to Active Directory from my hostmachine(where single node hadoop cluster is deployed using Ambari). Do you think the steps you mentioned to setup Kerberos configs (before enabling Kerberos) are aligned with my case?

avatar
Master Mentor

@tsharma

Can you attach the following files in here

- krb5.conf,

- kdc.conf and

- kadm5.acl.

What is your AD Domain?

Did you import the AD cert? self signed or CA ?