Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Impala Parallel query

Impala Parallel query

Explorer

Hi,

 

Is it possible to use Impala for web application for fast data fetching.

Actually I wan to use one of Impala query in web application.

I tested with simple query for hit through the application it's took just secs so it's good.

But when multipule people like some lakhs of people going to hit at a time, that time qeury is too slow and getting responce also too much late.

 

How to tune the impala for parallel processing with single query going to use multipule people.

 

Please help me into this.

 

Thanks

3 REPLIES 3

Re: Impala Parallel query

Master Collaborator

I'd recommend trying to understand your workload better to identify where the bottleneck is:

 

Are all cluster resources being utilized? If so, then you can't really go any faster.

 

If the cluster resources are not fully utilized then you may be experiencing hot spots. For example, if you issue all queries against the same impalad, then that impalad may become a bottleneck since it's the coorindator for all queries (which sometimes does actual work). You may also see hotspots if most queries access a small subset of the data, but very frequently the same data, e.g., always querying the most recent partition.

 

Understanding in which case you are in will help us guide you towards a potential solution.

Re: Impala Parallel query

Explorer

Thanks for replying alex.

 

Actually I am not aware of cluster side. I think my cluster will not utilizing the cluster resources.

 

I saw one of example like we can execute the query faster, before that we have to execute the below command

set disable_codegen=true;

 

I tried this example in shell its worked on below 1 sec. It's good.

Now here I would like to set this parameter permanently, when ever anybody hit any query through jdbc query should be execute below 1sec. 

 

Can yopu please help me into this how to set.

 

Thanks & Regards

Suresh V

Re: Impala Parallel query

Explorer

If any multipule users hit the query at a time, for each hit user have to get equal responce.

 

select * from x where ab = 'suresh'

select * from x where ab = 'dghdgn'

select * from x where ab = 'dgtyj'

.

.

 

 

So shall impala will handle parallel query.