CDH Snapshots Repository down?

[#1]

(Please forgive/correct me if this is the wrong board, I couldn't find one specifically about release/build issues)

As of Friday (10/31/2014) afternoon (~3pm PDT) I stopped being able to resolve dependencies from the Cloudera Maven repositories. At first I couldn't get Spark to resolve ("org.apache.spark" % "spark-assembly_2.10" % "1.1.0-cdh5.2.0-SNAPSHOT"). The dist tarballs were just missing. Around 4pm they mysteriously appeared again, however, I was then unable to resolve the cdh-root dependency. Looking through the poms it appears the only dist source is "http://maven.jenkins.cloudera.com:8081/artifactory/cdh-snapshot-local" which was down (and is still, as of 10am PST today).

Are there any other repositories I can reference? Is this sort of thing a regular occurance with (relatively) new releases?

For reference, here are the repositories I am currently referencing:

  "Cloudera" at "https://repository.cloudera.com/artifactory/public/",
  "cloudera-repos" at "https://repository.cloudera.com/artifactory/cloudera-repos/",
  "ClouderaRcs" at "https://repository.cloudera.com/cloudera/cdh-releases-rcs/",

Thank you for any help/pointers.

[#2]

You should only rely on released (i.e. non-SNAPSHOT) versions in your own projects. CDH 5.2.0 was released 14 Oct, so there are no longer SNAPSHOT versions of the artifacts.

See the CDH documentation on using maven for info on what the proper version string is for the component you wish to use: CDH 5.2.0 Maven artifact coordinates

[#3]

Thank you. Normally I wouldn't depend on a SNAPSHOT, however, the release appears to be missing spark-assembly. When/will this be added to the release artifacts?

[#4]

This is all I could find for spark-assembly in the cloudera-repos referenced in the documentation:

https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/spark/spark-assembly_2.10/

It is not listed in the CDH 5.2.0 Maven artifact coordinates.

[#5]

Additionally, the non-SNAPSHOT version of cdh-root also appears to point to the offline http://maven.jenkins.cloudera.com:8081/artifactory/cdh-staging-local

(from https://repository.cloudera.com/cloudera/cloudera-repos/com/cloudera/cdh/cdh-root/5.2.0/cdh-root-5.2...

Are these pom files incorrect or is there an ETA on when maven.jenkins.cloudera.com will be back online?

[#6]

maven.jenkins.cloudera.com is an internal repository used in our internal build and publishing process. It is currently online but it is not available outside of Cloudera's internal network.

[#7]

Thank you for the clarification on that, although it seems odd it should be referenced in the poms on public servers.

Do you have any suggestions on what the proper reference for spark-assembly-1.1.0-cdh5.2.0 would be?

[#8 · Accepted solution]

Current versions of Spark don't have a spark-assembly jar artifact (see for example maven central for upstream). The assembly is used internally by distributions when executing Spark.

Instead you should have a dependency for whichever part of Spark you make use of, e.g. spark-core.