Created on 05-15-2016 07:06 PM - edited 09-16-2022 03:19 AM
Never hit this before until upgrading to CDH 5.7.
I have very large table with hundreds of billions of records. It's partitioned by YYYY, MM, DD, HOUR. Understandably a full compute stats takes eons to the point of being mostly unusable, so have always done compute incremental stats on each HOUR partition after they are added and populated.
But now that errors out with the following error:
ERROR: AnalysisException: Incremental stats size estimate exceeds 200.00MB. Please try COMPUTE STATS instead.
Is there a way to change that limit, or disable this entirely - otherwise how is one supposed to do an incremental stats on an extremely large table now?
Thanks!