11-03-2014 10:08 AM
(Please forgive/correct me if this is the wrong board, I couldn't find one specifically about release/build issues)
As of Friday (10/31/2014) afternoon (~3pm PDT) I stopped being able to resolve dependencies from the Cloudera Maven repositories. At first I couldn't get Spark to resolve ("org.apache.spark" % "spark-assembly_2.10" % "1.1.0-cdh5.2.0-SNAPSHOT"). The dist tarballs were just missing. Around 4pm they mysteriously appeared again, however, I was then unable to resolve the cdh-root dependency. Looking through the poms it appears the only dist source is "http://maven.jenkins.cloudera.com:8081/artifactory/cdh-snapshot-local" which was down (and is still, as of 10am PST today).
Are there any other repositories I can reference? Is this sort of thing a regular occurance with (relatively) new releases?
For reference, here are the repositories I am currently referencing:
"Cloudera" at "https://repository.cloudera.com/artifactory/public/",
"cloudera-repos" at "https://repository.cloudera.com/artifactory/cloudera-repos/",
"ClouderaRcs" at "https://repository.cloudera.com/cloudera/cdh-releases-rcs/",
Thank you for any help/pointers.
11-03-2014 11:06 AM - edited 11-03-2014 11:07 AM
You should only rely on released (i.e. non-SNAPSHOT) versions in your own projects. CDH 5.2.0 was released 14 Oct, so there are no longer SNAPSHOT versions of the artifacts.
See the CDH documentation on using maven for info on what the proper version string is for the component you wish to use: CDH 5.2.0 Maven artifact coordinates
11-03-2014 11:59 AM
This is all I could find for spark-assembly in the cloudera-repos referenced in the documentation:
It is not listed in the CDH 5.2.0 Maven artifact coordinates.
11-03-2014 12:31 PM
Additionally, the non-SNAPSHOT version of cdh-root also appears to point to the offline http://maven.jenkins.cloudera.com:8081/artifactory/cdh-staging-local
Are these pom files incorrect or is there an ETA on when maven.jenkins.cloudera.com will be back online?
11-03-2014 12:36 PM
maven.jenkins.cloudera.com is an internal repository used in our internal build and publishing process. It is currently online but it is not available outside of Cloudera's internal network.
11-03-2014 12:47 PM
Thank you for the clarification on that, although it seems odd it should be referenced in the poms on public servers.
Do you have any suggestions on what the proper reference for spark-assembly-1.1.0-cdh5.2.0 would be?
11-03-2014 01:49 PM
Current versions of Spark don't have a spark-assembly jar artifact (see for example maven central for upstream). The assembly is used internally by distributions when executing Spark.
Instead you should have a dependency for whichever part of Spark you make use of, e.g. spark-core.