In that case - scheduling of remote reads - for Kudu it's based on distributing the work for each scan across nodes as evenly as possible. For Kudu we randomize the assignment somewhat to even things out, but it's distribution is not based on resource availability.   I.e. we generate the schedule and then wait for the resources to become available on the nodes we picked. I understand that reversing that (i.e. find available nodes, then distribute work on them) would be desirable in some cases but there are pros and cons of doing that. For remote reads from filesystems/object stores, on more recent versions, we do something a bit different - each file has affinity to a set of executors and we try to schedule it on those so that we're more likely to get hits in the remote data cache. ... View more

@parthk  You can definitely use sentry for RBAC type of style in Impala you don't really need Kerberos but it's highly advised to have Kerberos why??? If you know historical sentry has been the weakest link in the security architecture of Cloudera that's the reason it was dropped in favor of Ranger in the upcoming new offering CDP. Having said that sentry role-based access Control (RBAC) is an approach to restricting system access to authorized users whereas Kerberos using keytabs is like a biometric passport where the password is only know to the keytab and principal that allows a process (a client) running on behalf of a principal (a user) to prove its identity to a verifier (an application server, or just server) without sending data across the network that might allow an attacker or the verifier to subsequently impersonate the principal. Kerberos optionally provides integrity and confidentiality for data sent between the client and the server. You can safely build your cluster without Kerberos especially for self-study and development but not for production. There are 2 types of Kerberos setup MIT and AD Active Directory is a directory services implementation that provides all sorts of functionality like authentication, group and user management, policy administration and more in a centralized manner. LDAP (Lightweight Directory Access Protocol) is an open and cross-platform protocol used for directory services authentication hence the pointer in the Cloudera documentation to use LDAP/LDAPS HTH Happy hadooping ... View more

For now. I think the two possible methods I outlined might work. Additionally, you could export the data to something like parquet or avro, using Spark or Impala, and then reload the data in the new cluster. ... View more