Support Questions
Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Errors while running alter table and compute stats on tables with large number of partitions.


We are running Impala v2.7.0-cdh5.10.1 (876895d), and are facing similar issue.


While running alter table <table-name> recover partitions; - we get "Error communicating with impalad: TSocket read 0 bytes" error message. after multiple attempts to recover this table, we were finally able to recover partitions on the table.


At which point, we ran "compute stats <table-name>;" and received this error message: CatalogException: Table was modified during stats computation.



Although both these errors may be mutually exclusive, somehow they seems to only happen on the two tables with over 10,000 partition files.




Did anyone look into this issue?

I am also facing the same issue.I am using CDH5.10.2 


If you're seeing a crash, any diagnostics are useful. E.g. log files with crash messages from /var/log/impalad, hs_err_pid*.log files, etc.


You can also file a bug report on the Apache JIRA: - it's generally easier to attach things and track things.
The crash could be something like but I'm just speculating. Generally large numbers of partitions or files  can blow up the size of metadata and cause degraded performance.



Below is the error message i received .


1137 2017-11-07 03:55:11,536  [INFO ] There are no more tasks to run at this time
1138 Starting Impala Shell without Kerberos authentication
1140 Server version: impalad version 2.6.0-cdh5.8.4 RELEASE (build 207450616f75adbe082a4c2e1145a2384da83fa6)
1141 Invalidating Metadata
1142 Query: invalidate metadata
1144 Fetched 0 row(s) in 4.11s
1145 Query: use `DBNAME`
1146 Query: insert overwrite table Table partition(recordtype) select adid,seg,profile,livecount,
1147 count(distinct mc) as nofs,stbnt,1 from table1 where livecount<>0 group by adid,seg,profile,livecount,stbnt
1149 CatalogException: Table 'dbname.table' was modified while operation was in progress, aborting execution.