Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Running spark with a new version parquet


Running spark with a new version parquet

New Contributor
Hello everybody

I am currently running cloudera 5.13.1 with parquet libraries 1.5.0 and yarn and I would like to profit from major improvements provided parquet 1.10.0 (min max stats usage) without upgrading my current version to the newest spark or cloudera version (knowing that even the newest version of cdh do not use this last version avec parquet). Is there any way to do this?

Maybe by using some maven shading or by executing another spark installation on yarn besides the main one

Best regards,

Don't have an account?
Coming from Hortonworks? Activate your account here