The fix for this was to upgrade CDH from 5.5.1 to 5.13.3 ... View more

@chriswalton007   There are different types of latencies    Namenode RPC latency Journal Node FSync latency network latency etc   There are few points here, 1. Your network latency will vary based on the traffic in your cluster. It may create trouble during peak hours. 2. The latency issue may leads to follows: As we know, the master daemons will always wait for the update from child daemons in every few seconds, and master will consider the child is not available/dead in case of any delay and look for alternate. It is unnecessary unless it is a real issue with child. 3. As far as NN RPC is the concern, in a HA cluster, both active and standby NN has to talk each other and it should be in sync with in few seconds. If they are not in sync and if something went wrong on active NN, the standby become active but it may not be up to date and it will lead to confusion.   end of the day, every seconds are matter in a distributted cluster.    But in your use case, not sure you are going to use cloudera director if so, the link that you have shared says it will not allow to create a mixed cloud/on-premise cluster. But if you are going to use a different tool and it will allow you to configure the mixed cloud/on-prem then you can go ahead based on the below... 1. If you are going to try this for a non-prod environment first 2. if you have less work loads   ... View more

@martinbo,   As mentioned by others, there are some options to ease the management of users and groups.    Common ones are:   1 - SSSD, IPA, Centrify OS level integration so that application calls to the OS are handled by those apps to make queries to a central LDAP source.  This requires a good deal of configuration, but it is a robust, enterprise-grade solution   2 - Manage your group and passwd files with automation tools like puppet, chef, etc. (mod once, "push out" changes to all hosts)   3 - Configure LdapGroupsMapping in HDFS so that hadoop services will do group lookups directly to LDAP. NOTE:  If you intend on letting users run jobs directly on YARN, you will still need to create local users on each host with a NodeManager since contains require the os user to be present.     ... View more

@scratch28   You can use clouder navigator to generate this report   Login as full admin to CM    Cloudera Management Service -> Navigator Metadata Server -> Cloudera Navigator (menu) -> search for 'impala' -> choose from left side options. You can choose upto last 365 days (or) custom period ... View more