Support Questions
Find answers, ask questions, and share your expertise

Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

Solved Go to solution
Highlighted

Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

I'm trying to build a simple Spark Java application that pulls its dependencies from the HDP Releases repository. My project only depends on:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.5.2.2.3.4.0-3485</version>
    <scope>provided</scope>
</dependency>

Through a complex web of dependencies jetty-util version 6.1.26.hwx is required, but it is not found in any publicly visible repository. Where can I find this dependency so that I can build a Spark application that uses Hortonworks packaged jars? Is it best to exclude the jetty-util dependency from spark-core_2.10 and then explicitly add it to the project with a non-HWX version?

(This question is similar to https://community.hortonworks.com/questions/6256/j... but the solution posted there does not work for my scenario.)

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

Mentor
5 REPLIES 5
Highlighted

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

Mentor

@Vladimir Zlatkin why don't you build it using the solution from @stevel ?

Highlighted

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

Because that doesn't work as I'm not trying to build the hadoop-auth package. Setting '-Djetty.version=6.1.26' does nothing since that variable isn't used anywhere in my Spark application.

Highlighted

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

Mentor

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

That worked!

I was missing the extra HDP Jetty repo:

<url>http://repo.hortonworks.com/content/repositories/jetty-hadoop/</url>
Highlighted

Re: Unable to build Spark application due to missing jetty-util:jar:6.1.26.hwx dependency

Mentor

please accept the answer to close the thread.