question Re: Removing Spark1.6 in Archives of Support Questions (Read Only)

Removing Spark1.6

JTRexp — Fri, 16 Sep 2022 13:52:13 GMT

We're using CDH 5.12.1 currently, which ships with Spark1.6. We have deployed Spark2.3 on the cluster, which is the distribution that we're actively using, and is working fine.

However, this does mean that we've got Spark1.6 binaries on our servers. Our security scans have picked these up as a vulnerability and we'd like to go ahead and remove them.

I'm wondering if anyone has attempted something like this before? If so, do they have any advice regarding it? I was simply going to have a look at what Spark1.6 files there are, then write a script that looped through our cluster and removed those files.

If someone has a more "official" way of doing things, that would be preferable. I'm more than aware that my proposal wouldn't exactly be supported.

As a follow up, have the Spark1.6 binaries been removed from more recent CDH versions?

Re: Removing Spark1.6

SaraNab — Tue, 20 Nov 2018 12:37:54 GMT

- CVE-2018-8024: doesn’t affect Spark1.6. - CVE-2018-1334: fixed in CDH5.14.4, CDH5.15.1, CDH5.12.3 and CDH5.16.0. You can upgrade to one of these versions to resolve the issue.

Re: Removing Spark1.6

Yuexin Zhang — Fri, 10 May 2019 09:51:07 GMT

CVE-2018-1334 is fixed in CDH 5.15.1 and higher version.

Re: Removing Spark1.6

Yuexin Zhang — Mon, 13 May 2019 11:29:07 GMT

To be more acturate, technically, CVE-2018-1334 is fixed in CDH 5.14.4.

But there's a new issue been found with similar privilege escalation vulnerability, which is CVE-2018-11760. We fixed CVE-2018-11760 in CDH 5.15.1. So with CDH 5.15.1, you won't be affected by these two similar privilege escalation vulnerabilities.

Re: Removing Spark1.6

PatliGalli — Tue, 28 May 2019 18:24:18 GMT

Hi Sara, I run vulnerability scans and our scanner picking up Spark 1.6 banner from the following path- for

CVE-2018-8024 vulnerability, you did mention this vulnerability doesn't affect SPark 1.6 but didn't give detail reasons. This is where Qualys picksup the banner-

/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/spark/conf/spark-env.sh: line 75: /usr/appl/cloudera/java/jdk1.8.0_162: is a directory

Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.6.0
/_/

Also we have 2 versions of SPARK running- do you really need version 1.6 to run v 2.3.0

Can you please help, advise.