We are trying to setup a single Hadoop cluster to be used in a multibranch company in a secure manner.
We have two AD domains in the company:
UPNs are at both of them. We installed HDP and connected HDP to bratislava.triviadata.com AD, with Hadoop SPNs created under
bratislava.triviadata.com LDAP subtree.
We have set up Cross-realm trust, where bratislava.triviadata.com trusts UPNs from wien.triviadata.com.
Linux is configured using SSSD, where SSSD by default translates the UPNs in form
can login to Linux machines, and this michalklempa is different person than the michalklempa@bratislava...
Thats what we are trying to achieve in Hadoop environment, too.
In Hadoop, default behavior is to strip all the components of UPN, only the "michalklempa" remains as the username.
After consulting at user@hadoop mailinglist (http://mail-archives.apache.org/mod_mbox/hadoop-user/201801.mbox/%3Ce86b7bfc-670b-7d62-6907-dd6596e220ae%40gmail.com%3E)
we changed auth_to_local in Ambari to produce user names in form:
We noticed that INFO log is logged when submitting to YARN:
this fix should be in Hadoop 2.8.0 (according to https://issues.apache.org/jira/browse/HADOOP-12751) and we use 2.7.3 (HDP 2.6.x), the fix is there.
We are unable to find where to track such Hortonworks fixes or cherry-picks, is there any Hortonworks JIRA available on these?
Anyway, the HDFS/YARN works with usernames containing @. What we are curious is, are there any relevant information on how this will
work with other tools in the HDP?
Kafka (may work - https://community.hortonworks.com/content/supportkb/150076/how-to-specify-auth-to-local-rules-for-ka... we haven't tested yet)
Is the fix of HADOOP-12751 part on any Hortonworks effort, to make HDP use of user names in full form user@domain?
Or should we stick with using only user part as is recommended by many tutorials (eg https://community.hortonworks.com/articles/59635/one-way-trust-mit-kdc-to-active-directory.html - stripping the domain part)?
And what about Ambari and Ranger LDAP sync? Both of these sync with a single DC, is it possible to get users from both DCs into these tools?