question protobuf is installed on ML Runtime, but needs upgradation in Support Questions

protobuf is installed on ML Runtime, but needs upgradation

habibalsa — Wed, 29 Mar 2023 08:26:16 GMT

Hello,

I was trying to install feast python library for which protobuf upgraded version is required. But protobuf 3.19.4 is part of available packages in ML runtime addons, so installing protobuf upgraded version is not shown in pip list command.

Any suggestion on upgrading libraries which are part of ML runtime addons ?

Re: protobuf is installed on ML Runtime, but needs upgradation

Gopinath — Thu, 06 Apr 2023 07:20:25 GMT

Preinstalled packages can be upgraded based on the business requirement.
I am curios to know what is the runtime version being used? By any chance did you test in the latest available runtime version?

Re: protobuf is installed on ML Runtime, but needs upgradation

VidyaSargur — Tue, 11 Apr 2023 07:02:19 GMT

@habibalsa, Has the reply helped resolve your issue? If so, can you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future? If you are still experiencing the issue, can you provide the information @Gopinath has requested?

Re: protobuf is installed on ML Runtime, but needs upgradation

habibalsa — Sun, 18 Jun 2023 07:15:39 GMT

Thanks @Gopinath, for your reply. I have used the latest available runtime version for python 3.7, 3.8 and 3.9. There are some packages which are coming from cml runtime, not from pip install, these packages need upgradation. But honestly, it can happen many times, which is why, it is difficult to raise business requirement each time.

Re: protobuf is installed on ML Runtime, but needs upgradation

Gopinath — Mon, 19 Jun 2023 05:02:45 GMT

@habibalsa subjected package [irrespective whether it is installed by user or preinstalled] can be upgraded through pip. At times the upgrade/install of packages may fail due to need root access or overlap with kernel libraries file, in those case it is recommended to have a custom image with required packages/version combinations built.
https://docs.cloudera.com/machine-learning/cloud/runtimes/topics/ml-creating-a-customized-runtimes-image.html

Re: protobuf is installed on ML Runtime, but needs upgradation

habibalsa — Mon, 19 Jun 2023 05:11:52 GMT

Hi @Gopinath, using pip I upgraded the package version before. Concern was about CMLAddons packages overriding those upgraded package.

As I am not the system admin, your provided link won't help me much.

Re: protobuf is installed on ML Runtime, but needs upgradation

Gopinath — Mon, 19 Jun 2023 05:40:25 GMT

Thanks for your response @habibalsa
You may considering getting a custom runtime add-on created with the packages. Still you need to work with CML admin to have them added in to CML.
https://docs.cloudera.com/machine-learning/cloud/runtimes/topics/ml-custom-runtime-addons.html