Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

Incremental stats size estimate exceeds 200.00MB

avatar
Contributor

Never hit this before until upgrading to CDH 5.7.

 

I have very large table with hundreds of billions of records. It's partitioned by YYYY, MM, DD, HOUR. Understandably a full compute stats takes eons to the point of being mostly unusable, so have always done compute incremental stats on each HOUR partition after they are added and populated.

 

But now that errors out with the following error:

 

ERROR: AnalysisException: Incremental stats size estimate exceeds 200.00MB. Please try COMPUTE STATS instead.

 

Is there a way to change that limit, or disable this entirely - otherwise how is one supposed to do an incremental stats on an extremely large table now?

 

Thanks!

Who agreed with this topic