Kerberized HDP2.5 Sandbox image - Always the local unix user is used

peter_zende · ‎05-15-2017

Hi,

I set up Kerberos authentication on a HDP2.5 Sandbox image. I've created my keytab as follows:

sudo kadmin.local
add_principal username/sandbox.hortonworks.com@MYHOST.NET
xst -k my_username.keytab username/sandbox.hortonworks.com@MYHOST.NET
quit
sudo chown username:users my_username.keytab

When I ssh to the sandbox, I'm able to run MR jobs with this keytab. However, I'd like to submit jobs from my host to the guest machine. Using the same keytab (I pass it to the MR job, login is done from code) will always fail as I always fall back to the local unix user

Inside the Sandbox (OK) :

2017-05-12 08:02:00 UTC DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  228 - hadoop login
2017-05-12 08:02:00 UTC DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  163 - hadoop login commit
2017-05-12 08:02:00 UTC DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  177 - using kerberos user:username/sandbox.hortonworks.com@MYHOST.NET
2017-05-12 08:02:00 UTC DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  199 - Using user: "username/sandbox.hortonworks.com@MYHOST.NET" with name username/sandbox.hortonworks.com@MYHOST.NET
2017-05-12 08:02:00 UTC DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  209 - User entry: "username/sandbox.hortonworks.com@MYHOST.NET"
2017-05-12 08:02:00 UTC DEBUG [main] org.apache.hadoop.security.UserGroupInformation  851 - UGI loginUser:username/sandbox.hortonworks.com@MYHOST.NET (auth:KERBEROS)

From host: (NOK)

2017-05-15 10:31:52 CEST DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  228 - hadoop login
2017-05-15 10:31:52 CEST DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  163 - hadoop login commit
2017-05-15 10:31:52 CEST DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  177 - using kerberos user:null
2017-05-15 10:31:52 CEST DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  193 - using local user:UnixPrincipal: username
2017-05-15 10:31:52 CEST DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  199 - Using user: "UnixPrincipal: username" with name username
2017-05-15 10:31:52 CEST DEBUG [main] org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule  209 - User entry: "username"
2017-05-15 10:31:52 CEST DEBUG [main] org.apache.hadoop.security.UserGroupInformation  851 - UGI loginUser:username (auth:KERBEROS)

From this I assume that the username/sandbox.hortonworks.com@MYHOST.NET principal needs to be mapped to the username/sandbox.hortonworks.com@MYHOST.NET username in the hadoop.security.auth_to_local property.

I tried to create this mapping but when testing it with hadoop org.apache.hadoop.security.HadoopKerberosName username/sandbox.hortonworks.com@MYHOST.NET it always resulted in error.

My oririnal hadoop.security.auth_to_local looks like:

<property>
  <name>hadoop.security.auth_to_local</name>
  <value>
    RULE:[1:$1@$0](ambari-qa-sandbox@MYHOST.NET)s/.*/ambari-qa/
    RULE:[1:$1@$0](hbase-sandbox@MYHOST.NET)s/.*/hbase/
    RULE:[1:$1@$0](hdfs-sandbox@MYHOST.NET)s/.*/hdfs/
    RULE:[1:$1@$0](spark-Sandbox@MYHOST.NET)s/.*/spark/
    RULE:[1:$1@$0](spark-sandbox@MYHOST.NET)s/.*/spark/
    RULE:[1:$1@$0](zeppelin-sandbox@MYHOST.NET)s/.*/zeppelin/
    RULE:[1:$1@$0](.*@MYHOST.NET)s/@.*//
    RULE:[2:$1@$0](amshbase@MYHOST.NET)s/.*/ams/
    RULE:[2:$1@$0](amszk@MYHOST.NET)s/.*/ams/
    RULE:[2:$1@$0](atlas@MYHOST.NET)s/.*/atlas/
    RULE:[2:$1@$0](dn@MYHOST.NET)s/.*/hdfs/
    RULE:[2:$1@$0](falcon@MYHOST.NET)s/.*/falcon/
    RULE:[2:$1@$0](hbase@MYHOST.NET)s/.*/hbase/
    RULE:[2:$1@$0](hive@MYHOST.NET)s/.*/hive/
    RULE:[2:$1@$0](jhs@MYHOST.NET)s/.*/mapred/
    RULE:[2:$1@$0](knox@MYHOST.NET)s/.*/knox/
    RULE:[2:$1@$0](livy@MYHOST.NET)s/.*/livy/
    RULE:[2:$1@$0](nfs@MYHOST.NET)s/.*/hdfs/
    RULE:[2:$1@$0](nm@MYHOST.NET)s/.*/yarn/
    RULE:[2:$1@$0](nn@MYHOST.NET)s/.*/hdfs/
    RULE:[2:$1@$0](oozie@MYHOST.NET)s/.*/oozie/
    RULE:[2:$1@$0](rangeradmin@MYHOST.NET)s/.*/ranger/
    RULE:[2:$1@$0](rangertagsync@MYHOST.NET)s/.*/rangertagsync/
    RULE:[2:$1@$0](rangerusersync@MYHOST.NET)s/.*/rangerusersync/
    RULE:[2:$1@$0](rm@MYHOST.NET)s/.*/yarn/
    RULE:[2:$1@$0](yarn@MYHOST.NET)s/.*/yarn/
    DEFAULT
  </value>
</property>

My krb5.conf:

[libdefaults]
  renew_lifetime = 7d
  forwardable = true
  default_realm = MYHOST.NET
  ticket_lifetime = 24h
  dns_lookup_realm = false
  dns_lookup_kdc = false
  default_ccache_name = /tmp/krb5cc_%{uid}
[domain_realm]
  MYHOST.NET = MYHOST.NET
[logging]
  default = FILE:/var/log/krb5kdc.log
  admin_server = FILE:/var/log/kadmind.log
  kdc = FILE:/var/log/krb5kdc.log
[realms]
  MYHOST.NET = {
    admin_server = kdc1.dev
    kdc = kdc1.dev
  }

Could anyone please help me to correct this issue?

Thanks, Peter

amerissa · ‎05-24-2017

Try copying the /etc/krb5.conf to your local so your local machine can see the sandbox's KDC. After that kinit and try it