Errors while running alter table and compute stats on tables with large number of partitions.

[#1]

We are running Impala v2.7.0-cdh5.10.1 (876895d), and are facing similar issue.

While running alter table <table-name> recover partitions; - we get "Error communicating with impalad: TSocket read 0 bytes" error message. after multiple attempts to recover this table, we were finally able to recover partitions on the table.

At which point, we ran "compute stats <table-name>;" and received this error message: CatalogException: Table was modified during stats computation.

Although both these errors may be mutually exclusive, somehow they seems to only happen on the two tables with over 10,000 partition files.

[#2]

Did anyone look into this issue?

I am also facing the same issue.I am using CDH5.10.2 

[#3]

If you're seeing a crash, any diagnostics are useful. E.g. log files with crash messages from /var/log/impalad, hs_err_pid*.log files, etc.

You can also file a bug report on the Apache JIRA:

https://issues.apache.org/jira/browse/IMPALA - it's generally easier to attach things and track things.

The crash could be something like https://issues.apache.org/jira/browse/IMPALA-2648 but I'm just speculating. Generally large numbers of partitions or files  can blow up the size of metadata and cause degraded performance.

[#4]

Below is the error message i received .

1137 2017-11-07 03:55:11,536  [INFO ] There are no more tasks to run at this time
1138 Starting Impala Shell without Kerberos authentication
1140 Server version: impalad version 2.6.0-cdh5.8.4 RELEASE (build 207450616f75adbe082a4c2e1145a2384da83fa6)
1141 Invalidating Metadata
1142 Query: invalidate metadata
1143
1144 Fetched 0 row(s) in 4.11s
1145 Query: use `DBNAME`
1146 Query: insert overwrite table Table partition(recordtype) select adid,seg,profile,livecount,
1147 count(distinct mc) as nofs,stbnt,1 from table1 where livecount<>0 group by adid,seg,profile,livecount,stbnt
1148 WARNINGS:
1149 CatalogException: Table 'dbname.table' was modified while operation was in progress, aborting execution.
1150
1151