Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Impala - On-demand metadata

avatar
Expert Contributor

Hi,

I want to enable an on-demand metadata feature for impala as it gives a lot of improvement.

 

one of the features is that the cached metadata gets evicted automatically under memory pressure. 

 

I want to know if this feature can be enabled in Cloudera CDH 5.14.4 version.

 

Has anyone implemented this on prod on CDH 5.14.4 version?

1 ACCEPTED SOLUTION

avatar

On-demand metadata does not exist in C5.14.4. There was a technical preview version in C5.16+ and C6.1+ that had all the core functionality but did not perform optimally for all workloads and had some other limitations. After we got feedback and experience with the feature, we made various tweaks and fixes and in C6.3 we removed the technical preview caveat - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/impala_metadata.html and there and some important tweaks in patch releases after (i.e. 6.3.3).


It is enabled by default in the latest versions of CDP.


So basically if you want to experiment and see if it meets your needs, CDH5.16+ works, but CDH6.3.3+ or CDP has the latest and greatest.

View solution in original post

1 REPLY 1

avatar

On-demand metadata does not exist in C5.14.4. There was a technical preview version in C5.16+ and C6.1+ that had all the core functionality but did not perform optimally for all workloads and had some other limitations. After we got feedback and experience with the feature, we made various tweaks and fixes and in C6.3 we removed the technical preview caveat - https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/impala_metadata.html and there and some important tweaks in patch releases after (i.e. 6.3.3).


It is enabled by default in the latest versions of CDP.


So basically if you want to experiment and see if it meets your needs, CDH5.16+ works, but CDH6.3.3+ or CDP has the latest and greatest.