Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Impala Fetch taking extremely long time

Impala Fetch taking extremely long time

New Contributor

We have been using Impala for close to 1 year.  Recently I have noticed that when running queries that for some queries that query finsishes quickly but the fetch never completes even for a very small acount of data.

 

Where should I check to resolve this outstanding issue?

 

thanks

 

 

5 REPLIES 5

Re: Impala Fetch taking extremely long time

Contributor

I'd start by looking at the Query Timeline section of the runtime profile (It's near the top of the runtime profile).

The two counters that are interseting here are:

  ClientFetchWaitTimer: this is the time waiting in the client.

  RowMaterializationTimer: this is the time to construct the result rows

Re: Impala Fetch taking extremely long time

Explorer

Even I am facing the same issue and already had a look at the query profile, it says Client fetch wait timer is high but I don't understand why is it High? How can I further narrowed down the root cause? What can I see the solution.

 

I am not sure if we have the solution for this?

Re: Impala Fetch taking extremely long time

Cloudera Employee

ClientFetchWaitTimer measures the time when the coordinator had a rowbatch ready, but the client has not downloaded it. Having a high value here means that the client itself, or the network bandwidth to the client was the bottleneck of the whole process.

If you use impala-shell as client you can try the -B switch, which turns off pretty printing of the output and can significantly improve the throughput.

If you have a client app that you wrote then try to separate the fetching of the rows and processing of them and allocate more threads for processing.

 

Re: Impala Fetch taking extremely long time

Explorer

How do we separate the data fetch process within application? any ideas?

 

thanks

Suresh

Highlighted

Re: Impala Fetch taking extremely long time

Master Collaborator

In some cases RowsAvailable can be reported a little too early - there are a few specific cases where Impala *thinks* that it has a row ready to return, but actually was a bit too optimistic and it takes longer than expected to produce that first row. We have an improvement JIRA open to make this more intuitive https://issues.apache.org/jira/browse/IMPALA-7551

Don't have an account?
Coming from Hortonworks? Activate your account here