Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

CDH Snapshots Repository down?

avatar
Explorer

(Please forgive/correct me if this is the wrong board, I couldn't find one specifically about release/build issues)

 

As of Friday (10/31/2014) afternoon (~3pm PDT) I stopped being able to resolve dependencies from the Cloudera Maven repositories. At first I couldn't get Spark to resolve ("org.apache.spark" % "spark-assembly_2.10" % "1.1.0-cdh5.2.0-SNAPSHOT"). The dist tarballs were just missing. Around 4pm they mysteriously appeared again, however, I was then unable to resolve the cdh-root dependency. Looking through the poms it appears the only dist source is "http://maven.jenkins.cloudera.com:8081/artifactory/cdh-snapshot-local" which was down (and is still, as of 10am PST today).

 

Are there any other repositories I can reference? Is this sort of thing a regular occurance with (relatively) new releases?

 

For reference, here are the repositories I am currently referencing:

  "Cloudera" at "https://repository.cloudera.com/artifactory/public/",
  "cloudera-repos" at "https://repository.cloudera.com/artifactory/cloudera-repos/",
  "ClouderaRcs" at "https://repository.cloudera.com/cloudera/cdh-releases-rcs/",

 

Thank you for any help/pointers.

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Current versions of Spark don't have a spark-assembly jar artifact (see for example maven central for upstream). The assembly is used internally by distributions when executing Spark.

 

Instead you should have a dependency for whichever part of Spark you make use of, e.g. spark-core.

View solution in original post

7 REPLIES 7

avatar
Expert Contributor

You should only rely on released (i.e. non-SNAPSHOT) versions in your own projects. CDH 5.2.0 was released 14 Oct, so there are no longer SNAPSHOT versions of the artifacts.

 

See the CDH documentation on using maven for info on what the proper version string is for the component you wish to use: CDH 5.2.0 Maven artifact coordinates

 

 

avatar
Explorer

Thank you. Normally I wouldn't depend on a SNAPSHOT, however, the release appears to be missing spark-assembly. When/will this be added to the release artifacts?

avatar
Explorer

This is all I could find for spark-assembly in the cloudera-repos referenced in the documentation:

https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/spark/spark-assembly_2.10/

 

It is not listed in the CDH 5.2.0 Maven artifact coordinates.

avatar
Explorer

Additionally, the non-SNAPSHOT version of cdh-root also appears to point to the offline http://maven.jenkins.cloudera.com:8081/artifactory/cdh-staging-local

 

(from https://repository.cloudera.com/cloudera/cloudera-repos/com/cloudera/cdh/cdh-root/5.2.0/cdh-root-5.2...

 

Are these pom files incorrect or is there an ETA on when maven.jenkins.cloudera.com will be back online?

avatar
Expert Contributor

maven.jenkins.cloudera.com is an internal repository used in our internal build and publishing process. It is currently online but it is not available outside of Cloudera's internal network.

avatar
Explorer

Thank you for the clarification on that, although it seems odd it should be referenced in the poms on public servers.

 

Do you have any suggestions on what the proper reference for spark-assembly-1.1.0-cdh5.2.0 would be?

avatar
Expert Contributor

Current versions of Spark don't have a spark-assembly jar artifact (see for example maven central for upstream). The assembly is used internally by distributions when executing Spark.

 

Instead you should have a dependency for whichever part of Spark you make use of, e.g. spark-core.