Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark 2.1 artifacts in HDP maven repository

Highlighted

Spark 2.1 artifacts in HDP maven repository

New Contributor

Hi

I'm trying to use Spark 2.1 on HDP 2.6 technical preview.

From spark-submit I see the following version of Spark: scala> spark.version res0: String = 2.1.0.2.6.0.0-331

Where can I find this specific version of Spark to add it as a provided dependency?

I can't find it here: http://repo.hortonworks.com/content/repositories/releases/org/apache/spark/spark-parent_2.11/

(only 2.0.0.2.5.0.0-1245/ and 2.0.0.2.5.3.0-37/)

Should I use another HDP repo or original Apache Spark 2.1 artifacts?

1 REPLY 1
Highlighted

Re: Spark 2.1 artifacts in HDP maven repository

Explorer

Vitaly,

Looks like these are the maven artifacts for packaging the HDP distribution. I suggest using those from Maven Central: http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.spark%22

Don't have an account?
Coming from Hortonworks? Activate your account here