In the scenario that someone who previously had admin access to the cluster has left the organisation (and assuming they know all the database and other passwords and may still have access to the network), what is the best practice for changing the passwords?
We remove the user from LDAP and revoke the Kerberos ticket to disallow any logins there. We change the Ambari database password, the Hive metastore, Oozie, and Ranger (and any services which are backed by SQL Db) database passwords. We also need to change the TrustStore password. We remove the public key from `authorized_keys` on every account on every machine.
The question is, what have I missed, and how to achieve this with minimum cluster disruption?
Hi @Derwin McGeary I think your list is actually pretty complete, there are other elements such as HDFS Data Encryption Keys that at a minimum should probably be rotated if you're using it.
You also have the consideration as to whether you remove the user or just disable them, so you can still retain accurate history later (i.e. looking back and seeing "unknown user" performed configuration changes, vs "user-departed-disabled" performed configuration changes).
When it comes to this it's very much about following the account policies your organisation has in place.
I would also typically attempt as much as possible to reduce the scope of which admins can see things like db passwords etc. As you add more components to your cluster the picture gets more complex, so keep that in mind when you add new services to add new steps to the user decommissioning process.