question Any query fails on tables with lot of partitions in Archives of Support Questions (Read Only)

Any query fails on tables with lot of partitions

Minutis — Fri, 16 Sep 2022 08:54:44 GMT

[hostname:21000] > describe tableX;
Query: describe tableX
ERROR: AnalysisException: Failed to load metadata for table: default.tableX
CAUSED BY: TableLoadingException: TableLoadingException: Failed to load metadata for table: tableX
CAUSED BY: TTransportException: java.net.SocketTimeoutException: Read timed out
CAUSED BY: SocketTimeoutException: Read timed out

When I involve any table with many partitions (like tableX - 6k+ partitions) in any query it always fail with error "Read time out". Other queries run fine. I know that having a lot of partitions is not a good idea, but since doing anything about partition count isn't an option, is there anything I can do for now? I have tried REFRESH tableX, still, can't even execute describe tableX query.

USING IMPALA 1.2.3 + CDH 4.4.0

Re: Any query fails on tables with lot of partitions

alan — Thu, 27 Feb 2014 19:36:39 GMT

Hi,

Can you set

<property>
<name>hive.metastore.client.socket.timeout</name>
<value>3600</value>
</property>

in catalog service's hive-site.xml and restart the catalog service?

Thanks,

Alan

Re: Any query fails on tables with lot of partitions

Minutis — Mon, 03 Mar 2014 10:40:41 GMT

If by "catalog service's hive-site.xml" you mean /etc/imapala/hive-site.xml then I just tried it, restarted catalog service and it still gives me the same error message. And that message pops in an instance after pressing "enter" to submit query (describe tableX), like there is no timeout set.

Re: Any query fails on tables with lot of partitions

lskuff — Mon, 14 Apr 2014 15:42:26 GMT

Hi Minutis,
Have you tried upgrading to Impala v1.2.4? We made some enhancements in that release to fetch partitions in batches which should resolve your problem.

Thanks,
Lenni