Thanks Geoffrey Shelton Okot I have corrected krb5.conf and hadoop.security.auth_to_local as you suggested. listprincs also returned krbtgt/FDAQA.EYFIDS.NET@FDA.EYFIDS.NET. Regenerated all keytabs, restarted all stale components. But unforturnately, this didnt fix the issue. AD user is still not able to execute hdfs commands. Any clue is greatly appreciated! ... View more

Thanks @Geoffrey Shelton Okot I have corrected hadoop.security.auth_to_local as you suggested. But unfortunately that didnt fix the problem. The AD user is still not able to execute hdfs commands. Any clue is greatly appreciated! ... View more

Many thanks Robert. You are right. Unknowngly, I did kinit admin/admin@XYZDEV.ABC.NET earlier. That caused the problem. But now i did kinit bspchaseuser@XYZDEV.ABC.NET and i'm able to execute the HDFS commands succesfully. BUT ONLY if i add the principal for bspchaseuser manually in KDC server. Problem description: Not sure, where im making mistake. But the AD users can login and execute 'HDFS groups' command only if i manually add a principal for each of them with their username as shown below. More info: - Cluster is Kerberized with MIT KDC and integrated with AD using Winbind. - Everything is working, except i need to manually add principal for each AD user in MIT KDC server. krb5.conf in all nodes is as below: [libdefaults] renew_lifetime = 7d forwardable = true default_realm = FDADEV.EYFIDS.NET ticket_lifetime = 24h dns_lookup_realm = false dns_lookup_kdc = false default_ccache_name = /tmp/krb5cc_%{uid} #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 [logging] default = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log kdc = FILE:/var/log/krb5kdc.log [realms] FDADEV.EYFIDS.NET = { admin_server = dev-hdp-mitkdc.fdadev.eyfids.net kdc = dev-hdp-mitkdc.fdadev.eyfids.net default_domain = fdadev.eyfids.net } FDA.EYFIDS.NET = { kdc = dev-addc.fda.eyfids.net admin_server = dev-addc.fda.eyfids.net default_domain = fda.eyfids.net } As per the link https://community.hortonworks.com/articles/59635/one-way-trust-mit-kdc-to-active-directory.html AD trust with MIT KDC is established and tried by adding below rules also #RULE:[1:$1@$0](^.*@FDA\.EYFIDS\.NET$)s/^(.*)@FDA\.EYFIDS\.NET$/$1/g #RULE:[2:$1@$0](^.*@FDA\.EYFIDS\.NET$)s/^(.*)@FDA\.EYFIDS\.NET$/$1/g Can you please let me know how to get principals of AD users added automatically instead of manually adding the principal for each of the AD user in MIT KDC server. ... View more

Thanks Robert for your quick reply. I kept the hadoop.security.authentication property to 'Kerberos' now. We used Winbind to integrate AD with Linux hosts. That is working fine. Im able to see the users and groups with 'wbinfo -u' and 'wbinfo -g'. But when i log in as AD user and execute "HDFS groups", then its not showing the AD group of that logged in AD user. After logging in as AD user, when i do klist, its returning Default principal: admin/admin@XYZDEV.ABC.NET 1)Should i expect the return value to be bspchaseuser/bspchase@XYZ.ABC.NET ? //bspchaseuser is the AD user and bspchase is the AD user's AD group. XYZ.ABC.NET is the AD domain. 2)Do i need to manually create principals in KDC server for each of the AD user ? //There are 100's of users, practically difficult. I created for 1 user but didnt work. 3)Do i need to manually declare any new RULE under Ambari -> HDFS ->Configs -> hadoop.security.auth_to_local ? //I tried different ways in writing the rule, but didnt work. Can you please help with the exact rule to be kept? You have all details. I spent a week trying to fix this. Can you please guide me to the right channel. Thanks. ... View more

Thanks Robert for quick reply. Let me elaborate the problem description: Our HDP2.6 cluster is kerberized and integrated with LDAP. Now, We want all LDAP users to be able to login to our cluster and execute HDFS commands. When any AD user(Ex: bspchaseuser) logs into putty and executes any HDFS command, that AD user is treated as 'admin' by HDFS instead of his actual user name. Please see below. One thing i noticed is, If i change hadoop.security.authentication value from 'kerberos' to 'simple' in Ambari -> HDFS -> Configs -> Advanced core-site Then HDFS commands treat the logged in user correctly as AD user but not as 'admin'. Please help me, what is the resolution ? More info: Our AD Domain is in the format XYZ.ABC.NET Our KDC Domain and HDP cluster Domain are in the format XYZDEV.ABC.NET Ambari -> Admin -> Kerberos -> Realm is also XYZDEV.ABC.NET The extra principal Ambari created apart from service principals is, RULE:[1:$1@$0](.*@XYZDEV.ABC.NET)s/@.*// Do i need to manually create principals in KDC server for each of the AD user ? //There are 100's of users, practically difficult. I created for 1 user but didnt work. Do i need to manually declare any new RULE under Ambari -> HDFS ->Configs -> hadoop.security.auth_to_local ? //I tried different ways in writing the rule, but didnt work. Can you please help with the exact rule to be kept? You have all details. Thanks ... View more