New Contributor
Posts: 1
Registered: ‎11-05-2018

Running spark with a new version parquet

Hello everybody

I am currently running cloudera 5.13.1 with parquet libraries 1.5.0 and yarn and I would like to profit from major improvements provided parquet 1.10.0 (min max stats usage) without upgrading my current version to the newest spark or cloudera version (knowing that even the newest version of cdh do not use this last version avec parquet). Is there any way to do this?

Maybe by using some maven shading or by executing another spark installation on yarn besides the main one

Best regards,