We want to secure our HDP cluster with Kerberos but we don't have AD in the company. We are just using LDAP for users authentication. How can we secure our HDP cluster in this case ?
Is it possible to install an MIT KDC and configure it to use our LDAP for users authentication ? if yes, is there any documentation on how to do this ?
Ambari integrates with several KDCs. Currently the two most common and easiest to use are the MIT KDC and Active Directory. In the future Ambari will officially integrate with FreeIPA and there is experimental support in some later versions of Ambari for it, but you can still use FreeIPA if you wanted to manually mange your Kerberos identities.
The MIT KDC solution is pretty easy to install. All you need to do is install and configure the MIT KDC on some host that all hosts in the Hadoop cluster can access - this host doest not have to be in the cluster, just accessible. Then use the Ambari Enable Kerberos Wizard to configure the cluster for Kerberos.
I have scripts for several OSes that can help you install an MIT KDC. Attached is one for CentOS6 (and it should also work on CentOS7) - install-kdcsh.txt
You can find docs on the MIT KDC at https://web.mit.edu/kerberos/krb5-1.13/doc/admin/install_kdc.html and https://web.mit.edu/kerberos/krb5-1.13/doc/admin/conf_ldap.html
thank @Robert Levas for your reply. Does this mean that I have to re-create all my users in MIT KDC ? or is it possible to connect MIT KDC with LDAP and have access to my organisation users through MIT KDC ?
I am not too familiar with how the MIT KDC integrates with an LDAP server backend... but you should be able to get away with not recreating users. However, you may need to set principal names for each user after the KDC-to-LDAP integration is complete. I would backup my LDAP server before trying any of this.
Be sure to take a look at https://web.mit.edu/kerberos/krb5-1.13/doc/admin/conf_ldap.html.
One thing that I suggest is to make sure the writable DN points to a special LDAP container since Ambari will use the KDC to create cluster-specific user and service accounts; and you probably do not want these mixed in with your enterprise users. To help this, you might want to consider 2 KDCS - one for the enterprise users that the Hadoop cluster essentially has read-only access to and one for Ambari to manage the cluster-specific identities within. This configuration will require a trust relationship to be setup between the two KDCs (which is not difficult) but will allow for a nice separation between the Ambari/Hadoop identities and the enterprise user identities.
I have openLDAP at my home cluster and I use MIT KDC. I have created my service principals in kdc and users stay in openldap. This way I don't have to create users on each linux box on all five machines that I have.
If you want to create service principals in LDAP then you can follow the link Robert shared above.
That's exactly what I want to do. Service principals manager by MIT KDC and user manager at a global level by LDAP. DO you have any pointers on how to do this? is it described in HWX doc? I looked but without much luck
I built my cluster about three years ago. But here is a high level overview of how this works.
1. OpenLDAP is installed on one of my linux machines. MIT KDC could care less of OpenLDAP.. They work independent of each other in my setup.
2. MIT KDC is setup on one machine. This is where I have created all my service principals.
3. Assume, there is no Hadoop. Users login to five machines I have to their linux accounts using OpenLDAP. Everything works fine. This you should already have, as I understand.
4. Now you install Hadoop and configure it to use Kerberos. For that follow simple Hadoop Kerberos setup here.
5. Then for your service principals you will have auth_to_local settings as described here.
You can use script given by Robert in above answer or you can refer to below article for automated Kerberos configuration.
Once you are done with Kerberos stuff, you can refer this link for integrating MIT KDC with OpenLDAP.