Ambari 2.4 Kerberos with FreeIPA

[#1]

Ambari 2.4 Kerberos with FreeIPA

This tutorial describes how to enable Kerberos using a FreeIPA server for LDAP and KDC functions on HDP 2.5. The following assumptions are made:

• An existing HDP 2.5 cluster
• No existing IPA server
• There are sufficient resources to create an m3.medium VM to house the FreeIPA server
• DNS is already taken care of in the environment
• FreeIPA will run on RHEL/CentOS 7

Step 1: Setup FreeIPA Server

Install Entropy Tools

Certain operations like generating encryption keys host entropy for creating random data. A fresh system with no processes running and no real device drivers can have issues generating enough random data for these types of operations. Install the rng-tools package and start rngd to help with this issue:

yum -y install rng-tools
systemctl start rngd
systemctl enable rngd

Install FreeIPA Server

Install NTP and the FreeIPA software and start the NTP service:

yum -y install ntp ipa-server ipa-server-dns
systemctl enable ntpd
systemctl start ntpd

In order to use FreeIPA for domain resolution within the cluster, there are a few pieces of information that need to be collected:

• DNS servers for external lookups. These will be configured as "forwarders" in FreeIPA for handing off DNS resolution for external lookups.
• Reverse DNS Zone name. This is used for configuring reverse DNS lookups within FreeIPA. The FreeIPA server will calculate this based on the IP address and Netmask of the server if it is unknown.
• DNS domain to use for the cluster
• Kerberos realm to use for the cluster (by convention, usually the domain in uppercase)
• The hostname of the FreeIPA server
• The IP address to use for the FreeIPA server (if there is more than one on the host).

ipa-server-install --domain=example.domain.com \
    --realm=EXAMPLE.DOMAIN.COM \
    --hostname=ipaserver.example.domain.com \
    --ip-address=1.2.3.4
    --setup-dns \
    --forwarder=8.8.8.8 \
    --forwarder=8.8.8.4 \
    --reverse-zone=3.2.1.in-addr.arpa.

Enable PTR Record Sync

In order for reverse DNS lookups to work, enable PTR record sync on the FreeIPA server.

Get a list of the DNS zones created:

ipa dnszone-find --all | grep "Zone name"

For each of the DNS zones, enable PTR sync:

ipa dnszone-mod $zonename --allow-sync-ptr=true

Configure krb5.conf Credential Cache

HDP does not support the in-memory keyring storage of the Kerberos credential cache. Edit the /etc/krb5.conf file and change:

default_ccache_name = KEYRING:persistent:%{uid}

to

default_ccache_name = FILE:/tmp/krb5cc_%{uid}

Create a hadoopadmin user

In order to create users in FreeIPA, an administrative use is required. The default admin@REALM user can be used (password created during IPA server install). Alternatively, create a hadoopadmin user:

kinit admin@EXAMPLE.DOMAIN.COM
ipa user-add hadoopadmin --first=Hadoop --last=Admin
ipa group-add-member admins --users=hadoopadmin
ipa passwd hadoopadmin

Ambari also requires a group to be created called ambari-managed-principals. This group is not currently created by the Ambari Kerberos wizard. Create the group:

ipa group-add ambari-managed-principals

Because of the way FreeIPA automatically expires the new password, it is necessary to kinit as hadoopadmin and change the initial password. The password can be set to the same password unless the password policy prohibits password reuse:

kinit hadoopadmin@FIELD.HORTONWORKS.COM

Step 2: Prepare the HDP Nodes

First, disable the chronyd service since it interferes with NTP (which FreeIPA prefers):

systemctl stop chronyd
systemctl disable chronyd

Configure the HDP nodes to use the FreeIPA server for DNS resolution:

echo "nameserver $ipaserver_ip_address" > /etc/resolv.conf

All nodes in the HDP cluster must have the ipa-client software installed and be joined to the FreeIPA server:

yum -y install ipa-client
ipa-client-install --domain=example.domain.com \
    --server=ipaserver.example.domain.com \
    --realm=EXAMPLE.DOMAIN.COM \
    --principal=hadoopadmin@EXAMPLE.DOMAIN.COM \
    --enable-dns-updates

On the Amberi server node, install the ipa-admintools package:

yum -y install ipa-admintools

Step 3: Enable Experimental FreeIPA Support

Support for FreeIPA is not enabled by default in Ambari. You must enable the experimental functionality in Ambari before you can select FreeIPA as an option in the Kerberos wizard. In a browser, navigate to:

http://ambariserver.example.domain.com:8080/#/experimental

Check the box next to enableipa:

Step 4: Run the Kerberos Wizard

Run the Kerberos wizard from Ambari (Admin -> Kerberos -> Enable Kerberos). Select "Existing IPA" and verify that the prerequisites have been met.

Enter the appropriate information into the KDC page:

Click through to the Configure Identities page of the wizard. There is a bug in the name of the Spark principal that needs to be corrected. FreeIPA requires principal names to be in lower case, but ambari allows the cluster name to be in mixed case. If the cluster contains capital letters, the creation of the Spark principal will fail. To account for this, the principal names should all contain a reference to the toLower() function in the cluster name variable to ensure that capital letters are corrected before creating the principal.

Change the spark.history.kerberos.principal parameter to include the toLower() function:

Change from:

${spark-env/spark_user}-${cluster_name}@${realm}

To:

${spark-env/spark_user}-${cluster_name|toLower()}@${realm}

The rest of the Wizard should complete successfully.

[#2]

To change the password policy in FreeIPA use the following command in the server cli:

ipa pwpolicy-mod --maxlife=0 --minlife=0 global_policy

[#3]

Hi !

We installed our Freeipa server. And we faced some troubles. We don't know if it's because our server was instalesl with the French locale but Ambari try to search a terme "password" that it do not get in the ipa answer :

org.apache.ambari.server.serveraction.kerberos.KerberosOperationException: Unexpected response from ipa: -------------------------------------
Utilisateur « hadoop-060217 » modifié
-------------------------------------
  Identifiant de connexion: hadoop-060217
  Prénom: hadoop-060217
  Nom: hadoop-060217
  Répertoire personnel: /home/hadoop-060217
  Interpréteur de commande: /bin/sh
  Nom principal: hadoop-060217@DMZ.DOMAIN.TLD
  Principal alias: hadoop-060217@DMZ.DOMAIN.TLD
  Adresse courriel: hadoop-060217@dmz.domain.tld
  Mot de passe aléatoire: saT4=1BwNwjV
  UID: 1302800005
  GID: 1302800005
  Compte désactivé: False
  Mot de passe: True
  Membre des groupes: ipausers, ambari-managed-principals
  Clés Kerberos disponibles: True


	at org.apache.ambari.server.serveraction.kerberos.IPAKerberosOperationHandler.updatePassword(IPAKerberosOperationHandler.java:551)
	at org.apache.ambari.server.serveraction.kerberos.IPAKerberosOperationHandler.createPrincipal(IPAKerberosOperationHandler.java:337)
	at org.apache.ambari.server.serveraction.kerberos.CreatePrincipalsServerAction.createPrincipal(CreatePrincipalsServerAction.java:256)
	at org.apache.ambari.server.serveraction.kerberos.CreatePrincipalsServerAction.processIdentity(CreatePrincipalsServerAction.java:159)

In the java class, IPAKerberosOperationHandler, there is some references to "password"

org/apache/ambari/server/serveraction/kerberos/IPAKerberosOperationHandler.java

• line 548
• line 562

 543     try {
 544       ShellCommandUtil.Result result = invokeIpa(String.format("user-mod %s --random", principal));
 545       if (!result.isSuccessful()) {
 546         throw new KerberosOperationException(result.getStderr());
 547       }
 548       Pattern pattern = Pattern.compile("password: (.*)");
 549       Matcher matcher = pattern.matcher(result.getStdout());
 550       if (!matcher.find()) {
 551         throw new KerberosOperationException("Unexpected response from ipa: " + result.getStdout());
 552       } 
 553       String old_password = matcher.group(1);
 554       
 555       String credentialsCache = String.format("FILE:%s", fileName);
 556       Process process = Runtime.getRuntime().exec(new String[]{executableKinit, "-c", credentialsCache, principal});
 557       reader = new BufferedReader(new InputStreamReader(process.getInputStream(), StandardCharsets.UTF_8));
 558       stderr = new BufferedReader(new InputStreamReader(process.getInputStream(), StandardCharsets.UTF_8));
 559       out = new OutputStreamWriter(process.getOutputStream());
 560       
 561       String data = readData(reader, stderr);
 562       if (!data.startsWith("Password")) {
 563         process.destroy();
 564         throw new KerberosOperationException("Unexpected response from kinit while trying to password for "
 565                 + principal + " got: " + data);
 566       }         
 567       LOG.debug("Sending old password");

Does anyone know a workaround to allow Ambari complete installation with Freeipa ?

Reagrds.

[#4]

How can I set the "enableipa" flag on the experimental page, via command line or REST API? Thanks.

[#5]

Hi, you are right, the only solution is to change locale to english.

the code implementation to support multi-language is complex.