Impala - On-demand metadata

[#1]

Hi,

I want to enable an on-demand metadata feature for impala as it gives a lot of improvement.

one of the features is that the cached metadata gets evicted automatically under memory pressure. 

I want to know if this feature can be enabled in Cloudera CDH 5.14.4 version.

Has anyone implemented this on prod on CDH 5.14.4 version?

[#2 · Accepted solution]

On-demand metadata does not exist in C5.14.4. There was a technical preview version in C5.16+ and C6.1+ that had all the core functionality but did not perform optimally for all workloads and had some other limitations. After we got feedback and experience with the feature, we made various tweaks and fixes and in C6.3 we removed the technical preview caveat - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/impala_metadata.html and there and some important tweaks in patch releases after (i.e. 6.3.3).

It is enabled by default in the latest versions of CDP.

So basically if you want to experiment and see if it meets your needs, CDH5.16+ works, but CDH6.3.3+ or CDP has the latest and greatest.