Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Requesting prioritized load of table(s):

Requesting prioritized load of table(s):

Hello,

 

whenever i am running either select or insert query on Impala, it takes huge timing. after seeing the log i can see most of the times going in loading the table. i see message like this :

 

Frontend.java:808] Requesting prioritized load of table(s): comm_tran_data,comm_tran_info

 

I cant understand when every impalad have catalog with all the metadata then why its trying to load.

I am using Impala version 2.7,2.9 and 2.1. in all the version same problem.

 

can you suggest the solution for it.?? this problem is coming in PROD env.

7 REPLIES 7

Re: Requesting prioritized load of table(s):

Master Collaborator

Impala doesn't eagerly load all metadata into its cache, it only does so the first time that the table is referenced (or after a REFRESH or INVALIDATE METADATA that affects the table).

 

Loading metadata can take a long time if your table has a lot of partitions or files or if there is another query that is loading a large table.

Re: Requesting prioritized load of table(s):

After starting impala,I do invalidate metadata on all the impalad nodes so
should I still get this problem ?

Re: Requesting prioritized load of table(s):

Guru
If you run INVALIDATE METADATA often, you will get this issue more often as well. You should avoid running INVALIDATE METADATA, and try to use REFRESH when practical, or only INVALIDATE METADATA a particular table you need.

Do not try to refresh whole impala metadata too often!

Re: Requesting prioritized load of table(s):

On startup I do invalidate metadata. After that whenever any file is loaded
I run invalidate metadata for the table. I don't run refresh table because
it takes longer time than invalidate metadata table.

Re: Requesting prioritized load of table(s):

any updates on this issue please this is really very irritating. when i see catalog tab i see all the tables loaded there. but when the same table used in query it goes like :

as src where src.r = 1
I1029 15:35:41.023432 21809 Frontend.java:822] Requesting prioritized load of table(s): default.comm_simeod_raw

 

i just do refresh whenever any partition is added in the table.even if i run the query on old data same problem comes.

Re: Requesting prioritized load of table(s):

Guru
Does it happen all the time?

How many partitions and files in HDFS do you have for those tables that have the issue? How many tables affected? Are they all having large number of partitions and tables?

Re: Requesting prioritized load of table(s):

not all the times but most of the times, table size is not even 100 gb. there are 1200 partitions in the table. my question is when the table is loaded in catalog which i can see by going to that tab then what does it mean by loading ? and why its doing ?

Don't have an account?
Coming from Hortonworks? Activate your account here