How do HDFS Permissions work after Kerberos is enabled

[#1]

I use CDP Private Cloud Base 7.1.7 and just enabled Kerberos security. I followed the setup documentation but could not proceed further than this step <https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/security-kerberos-authentication/topics/cm-se...>.

In short, I lost "supergroup" access to hdfs. Here are details.

* I created an AD account mysuperuser@example.com and an AD group mysupergroup@example.com.
* After Kerberos is enabled, I changed dfs.permissions.superusergroup=mysupergroup, and restarted the cluster. Certainly,  "mysupergroup" and "mysuperuser" do not exist anywhere in Hdfs POSIX permission settings.
* I kinited mysuperuser@example.com, but got hdfs permission denied error. It looks like that Kerberos could not understand AD groups associated with the kinited account.
* Then I changed dfs.permissions.superusergroup=mysuperuser, restarted all services, but still got permission denied error.

I intended to use Ranger to manage HDFS resource permissions. I could not get Ranger properly installed due to the HDFS permission error. Ranger depends on Solr and Solr uses HDFS. Right now Solr gave me an HDFS access error (Java error) - Caused by: org.apache.hadoop.ipc.RemoteException: Permission denied: user=solr, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x.

I am trying to understand how HDFS permission works after enabling Kerberos but before Ranger is operational. Right now I can only access hdfs via kiniting the hdfs keytab file, which should only be used as a last resort.

Thank you.

Best regards,

[#2]

@james_jones @pajoshi Do you have some insights here? Thanks!

[#3]

@Seaport,  Let's address the kerberos issue before Ranger.

1. Can you kinit as hdfs user?  (on the NN with the hdfs keytab /var/run/cloudera-scm-agent/process/<a_number>-hdfs-NAMENODE/hdfs.keytab) 
2. Once you have a hdfs kerberos ticket, can you list directories?
3. Did you properly configure sssd, integrated with AD in example.com realm on ALL cluster nodes? For the HDFS issue you're seeing, the group mapping, via sssd, to user is required on the active NN, but eventually you need it working on all nodes.
4. If you run the command "id mysuperuser", is he in mysupergroup?

For the Solr issue, check the CM -> Solr -> Configurations -> HDFS Data Directory. It should be something like /solr. If it's correct, you need to selecting CM -> Solr -> Actions -> Create HDS Home Dir. Then restart Solr. Note that after you install Ranger, the service name, znode and HDFS Home Dir will change to something like /solr-infra. If you need Solr for your own data (not service infrastructure like Solr and Atlas), install a separate Solr instance after installing Ranger.

Good luck.