Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Any query fails on tables with lot of partitions

avatar
Contributor

[hostname:21000] > describe tableX;
Query: describe tableX
ERROR: AnalysisException: Failed to load metadata for table: default.tableX
CAUSED BY: TableLoadingException: TableLoadingException: Failed to load metadata for table: tableX
CAUSED BY: TTransportException: java.net.SocketTimeoutException: Read timed out
CAUSED BY: SocketTimeoutException: Read timed out

 

When I involve any table with many partitions (like tableX - 6k+ partitions) in any query it always fail with error "Read time out". Other queries run fine. I know that having a lot of partitions is not a good idea, but since doing anything about partition count isn't an option, is there anything I can do for now? I have tried REFRESH tableX, still, can't even execute describe tableX query.

 

USING IMPALA 1.2.3 + CDH 4.4.0

1 ACCEPTED SOLUTION

avatar
Contributor
Hi Minutis,
Have you tried upgrading to Impala v1.2.4? We made some enhancements in that release to fetch partitions in batches which should resolve your problem.

Thanks,
Lenni

View solution in original post

3 REPLIES 3

avatar
Cloudera Employee

Hi,

 

Can you set 

 

<property>
   <name>hive.metastore.client.socket.timeout</name>
   <value>3600</value>
 </property>

 

in catalog service's hive-site.xml and restart the catalog service?

 

Thanks,

Alan

avatar
Contributor

If by "catalog service's hive-site.xml" you mean /etc/imapala/hive-site.xml then I just tried it, restarted catalog service and it still gives me the same error message. And that message pops in an instance after pressing "enter" to submit query (describe tableX), like there is no timeout set.

avatar
Contributor
Hi Minutis,
Have you tried upgrading to Impala v1.2.4? We made some enhancements in that release to fetch partitions in batches which should resolve your problem.

Thanks,
Lenni