question Re: CDH Snapshots Repository down? in Archives of Support Questions (Read Only)

CDH Snapshots Repository down?

BigDataBen — Fri, 16 Sep 2022 09:11:30 GMT

(Please forgive/correct me if this is the wrong board, I couldn't find one specifically about release/build issues)

As of Friday (10/31/2014) afternoon (~3pm PDT) I stopped being able to resolve dependencies from the Cloudera Maven repositories. At first I couldn't get Spark to resolve ("org.apache.spark" % "spark-assembly_2.10" % "1.1.0-cdh5.2.0-SNAPSHOT"). The dist tarballs were just missing. Around 4pm they mysteriously appeared again, however, I was then unable to resolve the cdh-root dependency. Looking through the poms it appears the only dist source is "http://maven.jenkins.cloudera.com:8081/artifactory/cdh-snapshot-local" which was down (and is still, as of 10am PST today).

Are there any other repositories I can reference? Is this sort of thing a regular occurance with (relatively) new releases?

For reference, here are the repositories I am currently referencing:

"Cloudera" at "https://repository.cloudera.com/artifactory/public/",
"cloudera-repos" at "https://repository.cloudera.com/artifactory/cloudera-repos/",
"ClouderaRcs" at "https://repository.cloudera.com/cloudera/cdh-releases-rcs/",

Thank you for any help/pointers.

Re: CDH Snapshots Repository down?

busbey — Mon, 03 Nov 2014 19:07:21 GMT

You should only rely on released (i.e. non-SNAPSHOT) versions in your own projects. CDH 5.2.0 was released 14 Oct, so there are no longer SNAPSHOT versions of the artifacts.

See the CDH documentation on using maven for info on what the proper version string is for the component you wish to use: CDH 5.2.0 Maven artifact coordinates

Re: CDH Snapshots Repository down?

BigDataBen — Mon, 03 Nov 2014 19:55:51 GMT

Thank you. Normally I wouldn't depend on a SNAPSHOT, however, the release appears to be missing spark-assembly. When/will this be added to the release artifacts?

Re: CDH Snapshots Repository down?

BigDataBen — Mon, 03 Nov 2014 19:59:06 GMT

This is all I could find for spark-assembly in the cloudera-repos referenced in the documentation:

https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/spark/spark-assembly_2.10/

It is not listed in the CDH 5.2.0 Maven artifact coordinates.

Re: CDH Snapshots Repository down?

BigDataBen — Mon, 03 Nov 2014 20:31:07 GMT

Additionally, the non-SNAPSHOT version of cdh-root also appears to point to the offline http://maven.jenkins.cloudera.com:8081/artifactory/cdh-staging-local

(from https://repository.cloudera.com/cloudera/cloudera-repos/com/cloudera/cdh/cdh-root/5.2.0/cdh-root-5.2.0.pom)

Are these pom files incorrect or is there an ETA on when maven.jenkins.cloudera.com will be back online?

Re: CDH Snapshots Repository down?

busbey — Mon, 03 Nov 2014 20:36:47 GMT

maven.jenkins.cloudera.com is an internal repository used in our internal build and publishing process. It is currently online but it is not available outside of Cloudera's internal network.

Re: CDH Snapshots Repository down?

BigDataBen — Mon, 03 Nov 2014 20:47:38 GMT

Thank you for the clarification on that, although it seems odd it should be referenced in the poms on public servers.

Do you have any suggestions on what the proper reference for spark-assembly-1.1.0-cdh5.2.0 would be?

Re: CDH Snapshots Repository down?

busbey — Mon, 03 Nov 2014 21:49:14 GMT

Current versions of Spark don't have a spark-assembly jar artifact (see for example maven central for upstream). The assembly is used internally by distributions when executing Spark.

Instead you should have a dependency for whichever part of Spark you make use of, e.g. spark-core.