What is the suggested maximum number of hadoop.security.auth_to_local mappings in a cluster? Would several thousand mappings be unreal?
Several thousand rules seems to be excessive and may slow down some operations, not to mention really hard to maintain. Is there really a need to make that many granular rules?
@Robert Levas unfortunately yes. This particular environment is managed by an enterprise standard in house tool that syncs LDAP users/groups with the linux machines periodically. The enterprise is large and the group formats vary widely across the user base. The linux machines cannot be setup to sync with LDAP directly.
As far as I know, there isn't a limit to the number of rules that can be set in Ambari. However, the field in Ambari's database is finite and will eventually overflow. The field holds all properties for a given configuration type. So the auth_to_local value will not be the only data in the field. The actual size of the field varies depending on the database used to house Ambari's schema. For example in MySQL and Oracle, the field size should hold about 4Gb of data, where in PostgreSQL the field has no limit.
Regarding how the auth_to_local mapping fields are handled in the individual Hadoop services, I am not familiar eough with them to be able to comment on.