question Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency in Support Questions

Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

vzlatkin — Tue, 05 Jan 2016 04:10:40 GMT

I'm trying to build a simple Spark Java application that pulls its dependencies from the HDP Releases repository. My project only depends on:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.5.2.2.3.4.0-3485</version>
    <scope>provided</scope>
</dependency>

Through a complex web of dependencies jetty-util version 6.1.26.hwx is required, but it is not found in any publicly visible repository. Where can I find this dependency so that I can build a Spark application that uses Hortonworks packaged jars? Is it best to exclude the jetty-util dependency from spark-core_2.10 and then explicitly add it to the project with a non-HWX version?

(This question is similar to https://community.hortonworks.com/questions/6256/j... but the solution posted there does not work for my scenario.)

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

aervits — Tue, 05 Jan 2016 07:02:19 GMT

@Vladimir Zlatkin why don't you build it using the solution from @stevel ?

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

vzlatkin — Tue, 05 Jan 2016 07:10:39 GMT

Because that doesn't work as I'm not trying to build the hadoop-auth package. Setting '-Djetty.version=6.1.26' does nothing since that variable isn't used anywhere in my Spark application.

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

aervits — Tue, 05 Jan 2016 07:34:51 GMT

here's an answer for you https://community.hortonworks.com/questions/1383/intelij-maven-hdp-public-repos-to-getting-indexed.html

http://repo.hortonworks.com/content/repositories/jetty-hadoop/org/mortbay/jetty/jetty/6.1.26.hwx/

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

vzlatkin — Tue, 05 Jan 2016 08:15:37 GMT

That worked!

I was missing the extra HDP Jetty repo:

<url>http://repo.hortonworks.com/content/repositories/jetty-hadoop/</url>

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

aervits — Tue, 05 Jan 2016 10:21:29 GMT

please accept the answer to close the thread.