Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How does Impala handle heterogeneous hardware?

Solved Go to solution

How does Impala handle heterogeneous hardware?

Contributor

Suppose we have 8 DataNodes with 128 GB RAM each of which 64 GB is allocated to Impala and we add 8 more nodes of 256 GB and we intend to allocated 128 GB to Impala.

 

My concern is, will the coordinator be smart enough to know the mem_limit of each node it sends fragments to? Are there any other known issues that come up with such a configuration?

 

NOTE: Current CDH version is CDH 5.4.4 (Impala v2.2)

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How does Impala handle heterogeneous hardware?

Master Collaborator

Hi! Good question.

 

Today, Impala is not aware of the heterogeneity and will split the work evenly among all available nodes - regardless of how much cpu/memory those nodes have.

 

1 REPLY 1

Re: How does Impala handle heterogeneous hardware?

Master Collaborator

Hi! Good question.

 

Today, Impala is not aware of the heterogeneity and will split the work evenly among all available nodes - regardless of how much cpu/memory those nodes have.

 

Don't have an account?
Coming from Hortonworks? Activate your account here