Reply
New Contributor
Posts: 3
Registered: ‎10-09-2013

Table with Many Partitions has errors when queried.

[ Edited ]

other non-partitioned and smaller number partition tables still work.

 

This table has 6,000 partitions.

 

in hive:

hive> select * from table_name limit 10;
FAILED: SemanticException org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out

 

 

In Impala:

 

Query: select count(1) , to_date(dt) from table_name group by to_date(dt) order by to_date(dt) limit 10000
ERROR: AnalysisException: Failed to load metadata for table: table_name
CAUSED BY: TableLoadingException: Failed to load metadata for table: table_name
CAUSED BY: TTransportException: java.net.SocketTimeoutException: Read timed out
CAUSED BY: SocketTimeoutException: Read timed out

Cloudera Employee
Posts: 16
Registered: ‎08-01-2013

Re: Table with Many Partitions has errors when queried.

[ Edited ]

With that many partitions, it takes time to load them from the metastore.  Set hive.metastore.client.socket.timeout to a large number, like 300 (units are seconds).

 

You may also want/need to increase the heap size for the metastore.

 

New Contributor
Posts: 3
Registered: ‎10-09-2013

Re: Table with Many Partitions has errors when queried.

Thanks,

 

We will change it and when we are able to restart it I will let you know how it worked.

Explorer
Posts: 6
Registered: ‎06-10-2014

Re: Table with Many Partitions has errors when queried.

Hi,

I got the same problem, could you tell me how did you solve this problem? appreciate it~
New Contributor
Posts: 2
Registered: ‎08-22-2018

Re: Table with Many Partitions has errors when queried.

I am facing same issue, pls provide the solution.

Announcements