Impala - On-demand metadata

HanzalaShaikh — Fri, 16 Sep 2022 14:39:09 GMT

Hi,

I want to enable an on-demand metadata feature for impala as it gives a lot of improvement.

one of the features is that the cached metadata gets evicted automatically under memory pressure.

I want to know if this feature can be enabled in Cloudera CDH 5.14.4 version.

Has anyone implemented this on prod on CDH 5.14.4 version?

Re: Impala - On-demand metadata

Tim Armstrong — Wed, 14 Oct 2020 18:15:28 GMT

On-demand metadata does not exist in C5.14.4. There was a technical preview version in C5.16+ and C6.1+ that had all the core functionality but did not perform optimally for all workloads and had some other limitations. After we got feedback and experience with the feature, we made various tweaks and fixes and in C6.3 we removed the technical preview caveat - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/impala_metadata.html and there and some important tweaks in patch releases after (i.e. 6.3.3).

It is enabled by default in the latest versions of CDP.

So basically if you want to experiment and see if it meets your needs, CDH5.16+ works, but CDH6.3.3+ or CDP has the latest and greatest.

question Re: Impala - On-demand metadata in Support Questions

Impala - On-demand metadata

Re: Impala - On-demand metadata