Accessing hdfs from secure to non-secure cluster (hadoop 2.7 or below)

[#1]

In non kerberized cluster the user name that we provide will be used to resolve group name. User to Group mapping will be done based on the value configured in the property hadoop.security.group.mapping.

In kerberized cluster we will be passing kerberos principal instead of user name, now HDFS has to resolve the principle to a user name in order to perform User to Group mapping. This resolution of user name from kerberos principal is done based on the value configured in hadoop.security.auth_to_local.

The value for hadoop.security.auth_to_local will be configured properly in a kerberized cluster.

When the client is running in a kerberized cluster and trying to access non kerberized HDFS, kerberos principal is sent as username. When the username (kerberos principal in this case) is received by a non-secure NameNode, it tries to getShortName from the principal, when that happens, NameNode will try to apply the rules configured in hadoop.security.auth_to_local property.

Since the cluster is non-secure, the property hadoop.security.auth_to_local is not configured, so NameNode will try to apply DEFAULT rule. While applying DEFAULT rule, the realm from kerberos principal (username) is compared with the realm configured in krb5.conf file of non-secure cluster (if krb5.conf file is not present, then the value of default realm is empty string).

If the realm present in principal doesn't match with default realm, and there is no other rule to match, [java.lang.IllegalArgumentException: Illegal principal name xxxxxxx@EXAMPLE.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to xxxxxxx@EXAMPLE.COM] is thrown (this can be found in NameNode log)

This behavior is changed in HADOOP-12751 (hadoop 2.8), It doesn't throw NoMatchingRule exception anymore, instead NameNode logs "No auth_to_local rules applied to xxxxxxx@EXAMPLE.COM" and proceeds

In case of NoMatchingRule Exception in non-secure NameNode (when the username received is a kerberos principal), check krb5.conf file for default realm, if the file is not present, create a new file (default location is /etc/krb5.conf) and add the appropriate default realm in the file.