Reply
Explorer
Posts: 10
Registered: ‎04-03-2017

Re: Need help with Impala 2.8 on CDH 5.10 upgrade

Thanks Lars ,

 

looked for this parameter as part of impala configuration.

 

Added it to impalad parameters and its working with out error :-)

 

what this message mean :

WARNINGS: Too many partitions selected, doing full recomputation of incremental stats

 

did it compute all table partions or just ones without statistics ?

 

I use table without data so compute stats (without incremental) completed on the same time (previous post - 219.37 seconds)

 

[gc-test-impala28-02.c.bi-environment-1271.internal:21000] > COMPUTE INCREMENTAL STATS  dwh.dwh_events;
Query: compute INCREMENTAL STATS  dwh.dwh_events
+----------------------------------------------+
| summary                                      |
+----------------------------------------------+
| Updated 4560 partition(s) and 382 column(s). |
+----------------------------------------------+
WARNINGS: Too many partitions selected, doing full recomputation of incremental stats
Fetched 1 row(s) in 262.02s

 

Thanks

 

Alon

Explorer
Posts: 10
Registered: ‎04-03-2017

Re: Need help with Impala 2.8 on CDH 5.10 upgrade

Hi Lars , Tim

 

Coputing stats on specific partion works as it should in 5.10 and seesm to fix the problem .

 

5.10 with inc_stats_size_limit_bytes = 1073741824 ( 1GB)

 

[gc-test-impala28-02.c.bi-environment-1271.internal:21000] > COMPUTE INCREMENTAL STATS dwh.dwh_events partition (cluster_id=11,day=20170401);
Query: compute INCREMENTAL STATS dwh.dwh_events partition (cluster_id=11,day=20170401)
+-------------------------------------------+
| summary                                   |
+-------------------------------------------+
| Updated 1 partition(s) and 382 column(s). |
+-------------------------------------------+
Fetched 1 row(s) in 2.23s

 

 

5.9

-------

 

[gc-dp-pdpprd-data-04.c.bi-environment-1271.internal:21000] > COMPUTE INCREMENTAL STATS test_partitions.dwh_events partition (cluster_id=11,day=20170401);
Query: compute INCREMENTAL STATS test_partitions.dwh_events partition (cluster_id=11,day=20170401)
ERROR: AnalysisException: Incremental stats size estimate exceeds 200.00MB. Please try COMPUTE STATS instead.

 

Many Thanks :-)

 

Alon

Contributor
Posts: 65
Registered: ‎12-30-2015

Re: Need help with Impala 2.8 on CDH 5.10 upgrade

Yeap, you're right
Announcements